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EPRRI, funded by the U. S. Department of 
Education's Office of Special Education Programs, 
investigates the impact of new educational 
accountability systems on students with disabilities 
and on special education. EPRRI addresses the 
research needs of policymakers and other key 
stakeholders by identifying critical gaps in current 
knowledge, seeking promising strategies, and 
publishing Topical Reviews, Policy Updates, and 
Issue Briefs. The Institute is a joint venture of the 
Institute for the Study of Exceptional Children and 
Youth at the University of Maryland, the National 
Center on Educational Outcomes at the University 
of Minnesota, and the Urban Special Education 



Leadership Collaborative. 
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Emerging State-Level Themes: 
Strengths and Stressors in Educational Accountability Reform 



One of the most visible and controversial as- 
pects of educational reform in the United States 
today is the demand for public accountability 
for student learning at all levels of the education 
system. For years public school systems have 
measured success based primarily on inputs and 
programs created rather than the impact of these 
factors on academic achievement (Goertz & Duffy, 
2003). Recent reforms attempt to measure school 
success by what really matters— whether or not 
children are learning. Performance indicators in- 
clude student achievement (generally using state 
and local assessments), student grades, student 
attendance, and school retention and completion 
(Linn, 2000; Sebba et. al., 2000). An additional 
and crucial tenet of current accountability reform 
is that performance on selected indicators should 
have direct consequences to systems and to students 
(Heubert & Hauser, 1999). 

The Institute for the Study of Exceptional Chil- 
dren and Youth, University of Maryland (UM) and 
its partner organizations the National Center on 
Educational Outcomes at the University of Min- 
nesota (NCEO) and the Urban Special Education 
Leadership Collaborative at the Education Devel- 
opment Center (EDC), established this research 
institute to bring greater attention to the role of 
special education and students with disabilities in 
education policy reform. The Educational Policy 
Reform Research Institute (EPRRI) is conducting a 
five-year program of policy analyses, research, and 
dissemination involving policymakers, practitio- 
ners, parents/families, advocates, and consumers. 

The purpose of this topical review is to present 
EPRRI’s state-level qualitative findings collected 
during in depth interviews in our four core study 
states of California, Maryland, New York, and 



Texas. The data presented reflect the perspectives 
of state-level personnel who were involved with the 
creation and implementation of their state account- 
ability systems at the time that the No Child Left 
Behind Act was written and signed into law. This 
paper presents a variety of voices, from division 
directors to parent representatives, and attempts 
to reveal the strengths and stresses of including 
students with disabilities in educational account- 
ability reform. 

Students with Disabilities and Accountability 
Reform 

Legislation at both the state and federal level 
strongly supports the inclusion of students with dis- 
abilities in educational accountability reform. There 
are two primary federal mandates that emphasize 
the need to include these students in accountability 
reform. These are the 1997 reauthorization of the 
Individuals with Disabilities Education Act (IDEA) 
and the reauthorizations of the Elementary and Sec- 
ondary Education Act (ESEA) of 1994 and 2001, 
the latter being known as the No Child Left Behind 
Act (NCLBA). 

Accountability reform and IDEA 97. In 

1997 IDEA was reauthorized with amendments 
that require students with disabilities to participate 
in general state and district wide assessments of 
achievement, in part to reflect the fact that schools 
are accountable for the success of all students 
(Huefner, 1999). In recognition that students with 
disabilities may encounter problems accessing the 
assessment or demonstrating their knowledge and 
skills due to the impact of their disabilities, IDEA 
97 requires that assessment accommodations be 
provided to individual students as needed. An issue 
that arises in terms of appropriate accommodations 
relates to whether the proposed accommodation 
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threatens the validity of the student’s score on a 
standardized test. Students with disabilities who 
cannot participate in general achievement tests 
with accommodations must participate in alternate 
forms of achievement testing to be developed by the 
state education agency (SEA) or the local education 
agency (LEA). 

As part of the move toward public accountabil- 
ity, IDEA 97 requires SEAs to report to the public 
the number of students receiving special education 
services participating in regular assessments and 
the number participating in alternate assessments. 
The SEA must make performance (achievement) 
data available, provided that the data released are 
statistically sound and do not reveal the identity of 
the children involved. In addition, IDEA 97 requires 
states to identify performance goals and indicators 
to be reviewed by the Secretary of education and the 
public every two years. One of the required indica- 
tors is the performance of students with disabilities 
on state assessments; the others are dropout rates 
and graduation rates. 

Accountability reform, ESEA, and stu- 
dents with disabilities. In 1965, the ESEA was 
enacted into law “to provide financial assistance... to 
local educational agencies serving areas with con- 
centrations of children from low-income families 
to expand and improve their educational programs 
by various means, which contribute to meeting the 
special educational needs of educationally deprived 
children” (Public Law 89-10, Section 201, Elemen- 
tary and Secondary School Act, 1965). The goal of 
the law remains the same, but, as a brief review of 
the legislative history reveals, the means to achieve 
this goal have changed over time (Wenning, Herd- 
man, & Smith, 2002). In 1988 a new accountability 
system for Title I (then Chapter 1) schools was 



established that required LEAs to use average indi- 
vidual student gains on annual standardized norm- 
referenced tests to identify schools with ineffective 
programs. However, concerns over the impact of 
Title I including low expectations for disadvantaged 
students, an emphasis in instruction on basic skills, 
isolation from the regular curriculum and a reliance 
on procedural compliance rather that outcomes, 
provided the momentum for further change in the 
next reauthorization (Goertz & Duffy, 2003). 

The result of this concern was the Improving 
America’s Schools Act (IAS A, 1994), which re- 
flected the standards-based direction of educational 
policy in the early 1990s (Wenning, et al. 2002). 
IASA eliminated the annual testing requirement 
by substituting testing at least once within 3 grade 
spans, 3-5, 6-9, and 10-12. In addition, IASA re- 
moved federal guidelines for determining annual 
school performance, allowing each state to define its 
own adequate yearly progress. Among other report- 
ing requirements, IASA mandated that programs 
receiving federal funds under Title I disaggregate 
the performance of multiple student groups, includ- 
ing race and disability (Wenning, et al.) 

The drive for greater accountability and edu- 
cational equity is embedded in NCLBA, which 
also significantly increased the role of the federal 
government in state education policy. NCLBA aims 
to increase student achievement, improve schools, 
provide parents and the community with better in- 
formation and close some long-lasting and troubling 
achievement gaps between disadvantaged students 
and their peers (Cohen, 2002). It requires states to 
implement statewide accountability systems cov- 
ering all public schools and students and to meet 
specific deadlines on the scope and frequency of 
student testing, revamp their accountability sys- 
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terns and guarantee that every classroom is staffed 
by a teacher qualified to teach in his or her subject 
area. 

The requirements for standards and assessments 
are indeed rigorous, but largely build on the existing 
Title I requirements promulgated under the IASA. 
The performance-based accountability require- 
ments, for most states however, require them to 
chart a course into new and unfamiliar territory. Ac- 
countability systems must be based on challenging 
state standards in reading, mathematics, and later 
science, annual testing for all students in selected 
grades, and annual statewide progress objectives 
ensuring that all groups of students remain on a tra- 
jectory toward proficiency by 2013-14. Addition- 
ally, the same annual achievement objectives must 
be determined, met, and reported for subgroups, 
including students with disabilities, racial/ethnic 
minority groups, and students with limited English 
proficiency (LEP). 

According to the NCLB regulations, adequate 
yearly progress (AYP) is met when three condi- 
tions are satisfied. First, not less than 95 percent 
of all students and all subgroups in the school must 
participate in state assessments at the school level. 
Final regulations include the provision that the num- 
ber of students in a subgroup must be of sufficient 
size to produce statistically reliable results for the 
95 percent requirement to affect AYP. Second, all 
students and each subgroup of students must meet 
or exceed the objectives set by the state. A safe 
harbor provision at the school level allows a school 
to make AYP even if one subgroup fails to make 
the required progress if the number of students in 
that subgroup who are not proficient has declined 
by 10 percent and the subgroup has made progress 
on other academic indicators. Third, progress must 



be made toward increasing high school graduation 
rates and on another state determined academic 
indicator for elementary and middle schools. 

School districts and schools that fail to make 
AYP toward statewide proficiency goals will, over 
time, be subject to improvement, corrective action, 
and restructuring measures aimed at getting them 
back on course to meet state standards. Schools that 
meet or exceed AYP objectives or close achieve- 
ment gaps will be eligible for academic achieve- 
ment awards. 

NCLBA: Issues and challenges for students 
with disabilities. The requirement for disaggre- 
gated participation and performance data and com- 
parable progress for target groups is new to federal 
accountability reform. Graham-Keegan, Orr, and 
Jones (2002) suggest that it is to ensure that schools 
concentrate on improving the academic progress of 
all students rather than concentrating initially on 
the more high achieving affluent children, and to 
enable states to monitor their progress in closing the 
achievement gap. Wenning, et al., (2002) suggest 
that this provision attempts to “rectify distortions 
and variations in performance masked by the reli- 
ance on school-wide averages” (p.38). 

For students with disabilities the mandated 95 
percent participation rate will end their widespread 
exclusion from participation in large-scale state as- 
sessments (Goldhaber, 2002). Commentators have 
pointed out that when states had the discretion to 
make their own exemption decisions, the result in 
some states was the exclusion of up to 40 percent 
or even 50 percent of school-age students with dis- 
abilities from assessment participation (Thurlow, 
Nelson, Teelucksingh, & Ysseldyke, 2000). In a 
study by NCEO it appears that states are offering 
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a range of test accommodations and modifications 
that allow more students with disabilities to par- 
ticipate in state large-scale assessments. However, 
they now face the issue of how to ensure that these 
assessments generate valid data (Thurlow, Lazarus, 
Thompson, & Robey, 2002). 

Goertz (2002) points out that while more and 
more students with disabilities are being tested, 
states vary considerably in whom they include in 
their assessment accountability index and whose 
scores are reported publicly. Issues of test validity 
and construct-relevance underlie the decisions that 
states have made about who is tested on what and 
how, whose test scores are reported and how, and 
whose scores are included in accountability mea- 
sures (Goertz, 2002; Wenning, et al. 2002). 

The decision on subgroup size for reporting and 
AYP accountability purposes is left to the states. 
The size chosen for reporting and accountability 
will have implications for students with disabilities 
due to the small number of this target group in many 
schools, especially at the elementary school level 
and in rural schools. A high minimum “n” may mean 
that some schools will slip through the system, as 
it will not be possible to hold them accountable for 
the progress of students with disabilities. However, 
once these data are aggregated at the district level 
the situation may occur that although all schools are 
determined to have made AYP, the school district 
will not. In addition, some schools may not even 
report the scores of students with disabilities if the 
state has adopted a high minimum “n” for report- 
ing. Thus, it may not be possible for parents or the 
community to make informed decisions concerning 
this target group. 



The Purpose of the Education Policy Reform 
Research Institute (EPRRI) 

EPRRI is active in four core study states: 
California, Maryland, New York, and Texas. Within 
each state EPRRI staff and core state representatives 
identified and secured participation in the study of 
two school districts. EPRRI, in collaboration with 
core district representatives, identified individual 
schools in which to conduct further research ac- 
tivities. Five research questions frame EPRREs 
research activities at the state, district, and school 
level (Figure 1): 

Figure 1: EPRRI’s Guiding Research Questions 

1. How do broad education policies that 
incorporate high-stakes accountability 
include consideration of students with 
disabilities? 

2. What are the criteria to which special 
education has historically been held 
accountable? 

3 . What impact have educational account- 
ability mechanisms had on students 
with disabilities at the system and indi- 
vidual student levels? 

4. How are students with disabilities af- 
fected by educational accountability 
reforms? 

5. What changes could be made to better 
align special education with account- 
ability reform? 

EPRRI’s activities are conducted across three 
overlapping phases. Phase 1 focused on identifying 
the gaps in knowledge with respect to five specific 
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research questions. In Phase 2, which is ongoing, 
EPRRI is engaging in a high quality program of 
research conducted within four core study states and 
districts. Phase 3 consists of a strong, varied, and 
strategic dissemination program. During this study 
EPRRI has utilized both quantitative and qualitative 
research methodologies to investigate the impact 
of educational accountability systems on students 
with disabilities and special education programs. 
The purpose of this review is to present EPRRI’s 
state-level qualitative research findings. 
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Methods 



Research Design 

EPRRI conducts its research in four states: Cali- 
fornia, Maryland, New York, and Texas. When the 
study sites were selected, they varied across several 
key accountability features, including high-stakes 
versus low-stakes accountability consequences, 
recentness of reforms, stability versus instability 
of reform efforts, participation of students with 
disabilities in all accountability reports, and use of 
alternate assessments. However, since the incep- 
tion of the study, sites have moved into high-stakes 
accountability as defined by NCLBA. In addition, 
the states also reflect geographic and demographic 
diversity, including students with disabilities who 
are also members of minority, culturally and lin- 
guistically different groups as well. 

EPRRI is using “embedded case study meth- 
odology” involving within and cross-case analysis 
of extant data, interviews, site visits, focus groups, 
and document reviews. As Yin (1989) noted case 
study research can involve qualitative data only, or 
quantitative data only, or both. Although EPRRI’s 
overarching research design combines both data 
types, the data presented here are qualitative in 
nature. EPRRI researchers utilized two complemen- 
tary strategies to collect qualitative data: in depth 
interviewing and analysis of documents (Marshall 
& Rossman, 1999). The use of two methods of data 
collection and multiple data sources allowed for 
triangulation by data source. In addition, EPRRI 
staff developed a sound understanding of the his- 
tory and context of educational reform in each of 
the core study states. 

EPRRI staff conducts an ongoing review of 
individual state policies available on state web 
sites or directly from state participants. Examples 
of the types of documents reviewed include state 



board of education policies and minutes, strategic 
plans, reports from state Superintendents and Com- 
missioners, state department of education memos, 
state policies relating to standards based reform, 
assessments, and accountability, press releases, and 
state level reports. In addition, EPRRI staff review 
journals, newspaper articles, and legal cases when 
appropriate. 

The goals of the research project are to describe 
variations in policies, implementation strategies, 
issues, and impacts; to examine relationships be- 
tween certain decisions and effects on students and 
systems; and to examine state/district interactions 
that enhance or inhibit increased performance. We 
took the guiding questions for the research directly 
from the Office of Special Education’s grant prior- 
ity and were broken into subheadings and sample 
issues/indicators. In February 2001, the state-level 
participants and EPRRI staff together reviewed and 
revised a draft template of research questions. This 
process led to the development of 10 individual 
interview protocols focused on the areas of account- 
ability, assessment, monitoring, curriculum and 
instruction, special education, teacher certification 
and professional development, Title I, transition, 
and parent viewpoints. 

Participants 

Participants were 35 individuals from the state 
education agencies (SEAs) in EPRRI’s four core 
study states. The state special education director 
participated in the identification, initial contact, 
and interview arrangements with key personnel. 
We interviewed knowledgeable personnel from 
the following departments in each SEA: special 
education; accountability; testing; special education 
monitoring; Title I monitoring; curriculum; teacher 
certification; and professional development. We 
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sent a copy of the relevant interview protocol ahead 
of time to participants to familiarize them with the 
areas of specific interest to EPRRI. 

Procedure 

During the late Fall of 2001 and spring of 2002 
EPRRI staff conducted a series of interviews with 
key state level staff in all four core study states. 
The interviews were between one and two hours 
in length and adopted an open-ended response 
structure with the interview protocols acting as 
rough guides. Two interviewers were present, with 
EPRRI’s project manager present at all interviews 
to ensure continuity. The interviews were audio- 
taped with the consent of the participants so that 
the interview could take the form of a conversa- 
tion. Multiple researchers conducted the interviews 
(Eisenhardt & Bourgeois, 1988). This allows the 
researchers to overlap data analysis and data col- 
lection as recommended by Glaser and Strauss 
(1967) and Van Maanen (1988). The use multiple 
investigators provided complementary insights to 
add richness to the data analysis and enhance con- 
fidence in the findings (Eisenhardt, 2002). 

Data Analysis 

EPRRI researchers followed the qualitative data 
analysis procedures recommended by Miles and 
Huberman (1994). Field notes were written-up by 
individual researchers and added to the data corpus. 
Graduate research assistants transcribed the taped 
interviews verbatim, which produced approximate- 
ly 500 single spaced pages of data. Data analysis 
began with the creation of a contact summary sheet 
by EPRREs project director to develop an overall 
picture of the main points of each interview. The 
initial transcriber then read each contact summary 
sheet to identify bias and selectivity. We developed 



detailed descriptive write-ups for each site, based 
on the field notes, the contact summary sheet, and 
document reviews. This step in the data analysis 
is central to the generation of insight and helps 
researchers to cope with the enormous volume of 
data (Miles & Huberman, 1994; Pettigrew, 1988). 

We then entered the transcripts into a qualitative 
software program called Ethnograph, which allows 
for the analysis of text-based data into codes and 
categories of meaning. EPRRI researchers adopted 
a coding approach partway between the a priori 
and inductive approaches discussed by Miles and 
Huberman ( 1 994). A general coding scheme, based 
on that developed by Bogdon and Biklen (1992) 
was created to provide a structural, conceptual, and 
coherent order to the emerging codes. This coding 
scheme was based on the general domains covered 
by the research matrix and allowed codes to develop 
inductively, but enabled the researcher to “think 
about categories in which codes will have to be 
developed” (Miles & Huberman, p.g. 61). 

We developed clear operational definitions 
for each code so that the codes could be applied 
consistently. Code names closest to the concept 
being described were applied to the chunks of data. 
Initial coding of the data corpus was performed by 
the first author and a team of graduate researchers, 
who read and reread each interview line by line and 
coded the sentences or phrases relating to the par- 
ticipants’ perceptions of the effects of accountability 
on students with disabilities and the systems that 
serve them. EPRRI researchers followed the recom- 
mendations of Miles and Huberman (1994) relating 
to check-coding. Two researchers separately coded 
the first 8 pages of the opening interview from the 
first state visit and reviewed the coded sections to- 
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gether. Intercoder reliability was determined using 
the following formula: 

Reliability = number of agreements / total number 

of agreements + disagreements. 

Initial intercoder reliability at the state level was 
83 percent and rose to 91 percent after the differ- 
ences were clarified. A further check was performed 
two thirds of the way through the data analysis. 

A conceptually clustered matrix was developed 
during the early analysis based on the interview pro- 
tocols. The following decision rules were applied as 
themes were identified: First a theme was coded as 
present for a participant if it was mentioned repeat- 
edly or with strong emphasis during the interview. 
Second a theme was coded as present for a study 
site if mentioned by 2 or more participants. 

The second step in the analysis was to put the 
data back together again in a new way to reveal 
themes and stressors related to the impact of ac- 
countability at the state level. This process, simi- 
lar to axial coding in grounded theory (Strauss & 
Corbin, 1990) involved grouping and comparing the 
initial codes with each other and merging similar 
concepts together into larger encompassing themes. 
During this process, all key ideas, findings, and 
interpretations were presented and discussed by the 
EPRRI staff and core state representatives (Miles 
& Huberman, 1994). Original transcripts were used 
as evidence to support the emerging themes and at 
all times the researchers searched for examples that 
would contradict key findings (Yin, 1989). 

The next stage in the research was to conduct 
comparative or cross case analyses between each 
study site to enhance generalizability beyond each 



specific case and thus inform policy and practice. 
This stage of the research involves identifying simi- 
larities and constant associations to begin to form 
more general explanations (Ragin, 1987). 
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Results 



The state level interviews revealed many 
similarities and some differences in the ways that 
participants from the four core study states per- 
ceived the effects of accountability on students 
with disabilities. Although interviews in one state 
were conducted before the passage of NCLBA, 
state informants from all states were cognizant of 
its potential implications. Six major themes relat- 
ing to the impact of accountability reform on stu- 
dents with disabilities emerged from the state level 
data: 1) changing beliefs regarding students with 
disabilities; 2) increasing collaboration between 
regular and special education systems; 3) increas- 
ing alignment between state content standards and 
district instructional practices; 4) improving educa- 
tor capacity (improving the quality of new teachers 
and increasing the capacity of existing district and 
school personnel); 5) aligning special education 
accountability reform with general education ac- 
countability reform; and 6) meeting the technical 
challenges involved with including students with 
disabilities in general education reform. The fol- 
lowing section will demonstrate the complexities 
involved in implementing educational accountabil- 
ity reforms that include consideration of students 
with disabilities. 

Changing Beliefs Regarding Students with 
Disabilities 

We asked participants for their perspectives 
on the elements of accountability reform that had 
positive impacts on students with disabilities and 
those elements that they were concerned about. 
Participants across all study states identified two 
key aspects of accountability reform that they felt 
were working for students with disabilities: higher 
expectations for students with disabilities and the 



requirement to report performance results. Interest- 
ingly, only one informant expressed concerns that 
students with disabilities could be blamed if schools 
and districts failed to make AYP solely because of 
them. However, others were concerned that schools 
and districts may try to “hide” students with dis- 
abilities once their scores counted in accountability 
systems. 

High expectations for all students. One of 

the main premises of performance-based account- 
ability is the requirement that schools hold high 
expectations for all students. Informants from all 
four states agreed that expectations for students 
with disabilities were too low. For example, this 
participant pointed out that educators, both special 
and general, underestimate the abilities of students 
with disabilities and offer instructional programs 
that virtually ensure that students with disabilities 
meet these low expectations. For this representa- 
tive, performance-based accountability reform had 
the potential to break this cycle and allow children 
with disabilities the opportunity to demonstrate 
their abilities: 

So I think historically, if you learn from history, 
that we underestimate the ability of children with 
disabilities generally. We usually set up our instruc- 
tional program around those estimates and expec- 
tations, which mean that they are almost always 
lower than they should be. What we want to do is 
force people to reconsider their expectations and 
then get on with allowing children to have opportu- 
nities they haven’t had before. 

Another participant from the same state ex- 
pressed surprise at the high level of performance of 
students with disabilities on the state’s examinations 
and interpreted these findings as verification that 
low expectations are partly to blame for the poor 
performance of students with disabilities: 
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If you had asked me two years ago, I never would 
have told you that double the number of kids this 
year passed the new exam than took them 2 years 
ago. 1 think that right now what it says is that there 
were an enormous number of kids out there for 
whom expectations were unbelievably low, and that 
in fact what they are telling us is “I can do this.” 



A participant from a different state also identi- 
fied low expectations as a reason for the poor perfor- 
mance of students with disabilities, especially those 
from minority groups. This participant echoed the 
belief of other participants that performance-based 
accountability reform gave students with disabilities 
the chance to be successful and prove wrong those 
who underestimated their skills and abilities: 



are going to see really measured efforts to improve 
those scores. More and more people are less and 
less satisfied with poor results. 



Several participants from across the four study 
sites explained that the reporting of student scores 
was a key component of improving outcomes for 
students with disabilities. This participant explained 
how he/she had originally not supported public 
reporting, but now did: 



I was against public reporting, but I really believe 
that is the only way-that’s the wake up call, unfor- 
tunately. Once those scores are posted, that’s when 
we get the calls. It isn’t until those scores are made 
public that people are beginning to say, ah, here’s 
where we need to focus. 



You know when folks around this table asked me 
what causes this (poor performance for students 
with disabilities). I was scared, but I said it’s a lot 
of things, including low expectations of general 
education. And even though they have the problem, 
we contributed to it, so it’s our problem too. The 
one thing you can’t account for, how do you write a 
policy for teacher bias? Other than the fact that we 
continue to prove that kids can be successful. Those 
that say it can’t be done are usually interrupted by 
someone doing it. 



Reporting requirement. Participants across 
all study sites expressed the belief that reporting 
the results of students with disabilities on state 
assessments would lead to better outcomes. One 
participant commented that once it became clear 
how badly some students were doing schools would 
have to offer better instruction: 



We are hoping that one of the results of all of this 
data and obviously much more clearer and infor- 
mative data now than we have ever had is going 
to push people to improve their instructional pro- 
grams. People didn’t know before how badly some 
of those kids were doing and now they do. They 
are just starting to understand that now and it’s not 
really acceptable to a lot of people. Some superin- 
tendents are very unhappy with this stuff, but they 
didn’t even know about it until recently. I think you 



A participant in a different state commented 
on the role of the media in giving a high profile to 
accountability reform: 

I think we have a very aggressive media in terms of 
education issues. The first year where special edu- 
cation students’ results mattered there were some 
campuses where the exemption rate went from 0% 
to a very high percentage and we got a lot of phone 
calls from newspapers across the state. The papers 
really do look at this stuff and again whether they 
are special ed. friendly or sensitive or not, special 
education students are a big part of the media atten- 
tion focused on school districts. 

This participant, who was from the same state, 
echoed these sentiments and commented: “The 
state public accountability system and the public- 
ity that goes with it does more than these federal 
programs to bring a school out of the low perform- 
ing category.” 

A participant from a different state explained 
the effect that reporting and comparing scores by 
school and district had on parents of students with 
disabilities. According to this participant, making 
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information available to stakeholder groups pro- 
vides an impetus for change: 

The capital newspaper puts in all the scores of every- 
body in comparison of district by district, including 
special education scores. So I’ve actually had par- 
ents who know me come up next to me and say ‘did 
you see how that school’s special education student 
graduation rate is twice as high as mine?’ I say well 
that’s interesting, what do you think? And they 
say ‘well I’m going to go to the next school board 
meeting and ask them.’ And that’s what people are 
doing. I’ve heard this in states that haven’t been 
through it yet or are just starting, how the press is 
going to beat everybody up, but it isn’t beating ev- 
erybody up, it’s more a public information service 
that people are using smartly. People are asking 
what are you going to do to improve. And then the 
next day the Superintendent announces a new ini- 
tiative. When the assessment scores come out every 
single newspaper in the city will publish them. 

A participant from a different state articulated a 
similar view of the effect that publishing results had 
on schools and districts. This participant perceived 
that publishing results had the effect of changing 
behavior, of bringing districts together and encour- 
aging dialogue on how to improve: 

Last year we published for the first time every dis- 
trict in the state that has more than 200 kids. We 
had a few excited superintendents. They’re paying 
attention now. The other interesting piece is that 
we’re showing them if they have lousy dropout 
rates there’s a comparable district that has a great 
track record and they’re starting to talk to each 
other. Just publishing stuff changes how people do 
their jobs. 

Interestingly, another participant from the same 
state pointed out that though reporting was a good 
idea, making improvement of the scores binding 
for all groups reported may have the unforeseen 
consequence of denying funds to schools that re- 
ally need them: 



The requirement of comparable improvements 
would actually serve to disadvantage some of the 
various schools that you are seeking to help. The 
compromise solution to this, of course, is to report 
it out without making a binding and 1 think, I mean 
that solution has always intrigued me but the down- 
side is we certainly want to steer away from any- 
thing that’s getting into a blame game. 



Hiding students with disabilities. Several state 
informants admitted to concerns that schools and 
school districts, because of the expectation that stu- 
dents with disabilities would not score very highly 
on state assessments, may try to “hide” students 
with disabilities. Interestingly, one of our study 
states already had an accountability system very 
similar to that legislated by NCLBA. We asked 
special education personnel in this state what their 
experiences had been with the state accountability 
system. State informants explained that initially the 
scores of students with disabilities were not includ- 
ed in the school accountability ratings. However 
this situation later changed. Several participants 
described what happened and how schools and 
school districts responded. One participant pointed 
out some school districts in the state responded to 
the state’s decision to include students with dis- 
abilities in accountability indices by exempting 
large numbers of students with disabilities from 
taking the state assessment: 



In 1999 the special education exemption rates rock- 
eted. You might interpret that as, well they are trying 
to hide their special education students, or they are 
not trying to do what’s best for them. But many of 
our school districts were distressed because the rea- 
son they had their special education students taking 
the state assessment was in hopes that at some point 
they would exit special education and be prepared 
to take the high school assessment. They make a 
strong case: we have the best interests of the child 
at heart, but now you are forcing us, because of our 
district rating, to protect ourselves instead of doing 
what’s right for students. 



The Educational Policy Reform Research Institute 



13 




eprn 



On the same issue, another participant com- 
mented that districts did not want to injure students, 
but that because the district rating was at risk the 
stakes were very high: 



It’s a balance all the way along. I don’t think they’re 
deliberately out there trying to hurt students. Now 
we’ve got a very high stakes accountability system 
and when push comes to shove, they chose their 
district rating over their philosophy of doing what’s 
best for students. 



Another state was in the process of clarifying 
its policy to school districts regarding the IEP team 
and its role in determining how students with dis- 
abilities should participate in state assessments. 
Several informants in this state described their 
concerns over the number of special education stu- 
dents “exempted” from participation in the state’s 
assessment by individual IEP teams: 



We did a comparison of the percentage of kids that 
were reported special education, you know in the 
fall and the number of kids that actually took the 
state test. There are some schools that exempt a 
large percentage of their kids from being tested. 
And that issue has been taken up by the special edu- 
cation division. They are working with schools that 
have a high rate of exemptions. 



When asked if the IEP team made the decision 
to exempt a student with a disability this participant 
confirmed this and added that it was possible that 
the schools themselves may have pressured the IEP 
team to exempt students with disabilities from the 
state assessments: 



will give us a lot more information about the whole 
school population and how these different exemp- 
tions are affecting how they look and whether it’s 
legitimate. 



Another participant from this state explained 
that from 2003 onwards there would be no IEP 
exemptions and that the IEP team should determine 
only the type of assessment used or the nature of the 
accommodations required and not whether the stu- 
dent with a disability participated in assessments: 



We’ve tried to be clear in state assessment direc- 
tions that an IEP team may not exempt a student 
from any of the tests. That the role of the IEP team 
is to identify how the student will participate and 
that that participation may be the test with no ac- 
commodations, the standard accommodations, the 
non-standard accommodations or on the alternate. 
But that we do expect that the student will be as- 
sessed. The only legitimate exemptions are the par- 
ent requests and those parent requests are open for 
any parent - disabled student or nondisabled stu- 
dent. 



Several participants from one state explained 
that the state was also monitoring the number of 
students with disabilities assessed using the state’s 
alternate assessment. Unlike many alternate assess- 
ments, this one was not designed for students with 
severe disabilities, but rather for those students with 
disabilities who were receiving instruction in the 
general curriculum at a lower level than their same 
age peers. Several participants expressed concern 
that the number of students assessed by the alternate 
assessment would be too high: 



We are concerned about that, because there are 
some high schools, for instance, that all their kids 
with disabilities were exempted and that didn’t 
seem very likely. So this year, 2002 what we are 
doing is, we’re having every student enrolled in 
the district will have the test header sheet submit- 
ted. There is a space on there to say that they are 
exempted IEP, a parent opt out or they just didn’t 
end up taking the test for whatever reason. And that 



I think that we are going to see a higher percentage 
of students taking alternate than we are comfort- 
able with. So that will be the next thing to look at, 
through monitoring look at the decision-making 
process from the state level to the school level. Re- 
ally focusing on why special education students 
who are in the regular general curriculum with an 
accommodation of extended time are not taking the 
regular assessment. 
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Another participant from the same state reiterated 
this concern: 

Some of our future challenges with the new law will 
be the need to monitor those decisions about how 
many students tested with the regular assessment 
and how many tested with the state’s alternate. We 
worry about the impact at the local level given the 
media attention and the already high stakes nature 
of the state’s accountability system that too many 
students will be tested with the alternate that really 
meet the board requirement to take the regular as- 
sessment. So that will shift for us rather than mon- 
itoring for too few students tested to are schools 
testing appropriately. 



This participant went on to say that the situation was 
changing largely due to the changes in personnel 
that allowed the assessment division and the special 
education division to work together and combine 
their knowledge and skills: 



In the past we didn’t have as friendly a relationship 
with our assessment people as we do now, but when 
|name removed] came, he/she really is a breath of 
fresh air, he/she allows our folks to work together, 
and the special education division has some very 
strong advocates who understand assessment and 
test instruction and validity and all that stuff. So 
they should have been working together all along. 



Increasing Collaboration between Regular and 
Special Education Systems 

Data from across all study sites revealed that 
special education personnel were not always an 
integral part of state accountability reform initia- 
tives and that the needs of students with disabilities 
were not always considered in the formulation of 
policy. Comments from several participants reflect 
a strong sense that special education had been on 
the periphery of the reform agenda in the four 
study states, but that this situation was changing. 
For example, one participant pointed out that the 
division of special education was not even in the 
same building as the rest of the state department of 
education and historically had not been an integral 
part of previous policy formation in education: 

You notice we’re out in the portable. We are not in 
the main office. I told the state superintendent that 
I’m out in a portable and my digs are nicer than 
hers, however, it’s not okay to be out in a portable 
and maybe we should get a re-locatable and put it in 
the grass in front of the State Office of Education. 
There’s been the general education program, some- 
times they’ll let the at-risk folks come to the table, 
sometimes they’ll let the English language learner 
folks come to the table, and very infrequently did 
they ever let special education come to the table. 



Another participant from a different state 
described a similar situation in terms of physical 
and programmatic separation from the rest of the 
department of education: 



I’m a special educator who converted to the other 
side and I keep saying we can’t be separate any 
longer. Please come over and see how we are still 
in two buildings and we talk about these kids and 
those kids and our kids. 



A participant from a third state attempted to ex- 
plain the complex nature of the problems involved 
in merging special education and regular education 
policy. This participant pointed out that it required a 
balancing act because usually the division of special 
education wanted to be part of the general educa- 
tion initiatives from the beginning, but occasionally 
adopted a ‘wait and see’ approach. Interestingly, 
this participant also identified the importance of 
personal relationships in ensuring that special edu- 
cation remain involved in educational reform, but it 
is clear that this remains an ongoing issue: 



Our goal has been to try and latch onto many of 
the initiatives that are going on for all kids. Just 
be part of them... we’re not an afterthought, we 
are not an add on, we are part of it from the begin- 
ning if possible. That’s not always been easy to do. 
In fact there have been some projects they really 
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needed to get fixed before I wanted us to be part 
of it, but when they got it right we wanted in as 
quickly as we could. We’re not over the hump yet. 
I’m lucky because I’ve got so many people who 
trust me. There are still some folks that don’t get 
it. It’s not that we are an afterthought, we’re not a 
thought at all. 



We asked state informants whether their state’s 
content standards included accommodations for 
diverse student populations and the extent to which 
the teachers were aware of these curriculum frame- 
works. Participants from all states commented that 
their state’s content standards included students 
from diverse backgrounds generally by including 
stakeholder groups in the decision-making process. 
For example, one participant described how each 
content area committee included a representative 
from special education: 



We tried to make sure that there were people on 
every writing team that had a background with spe- 
cial needs students and gifted students within the 
state. Another thing we did was to create a separate 
group including people from the special population 
areas whose job was to look across all the subject 
areas for vertical and horizontal alignments. They 
recommended revisions, additions or changes that 
needed to be made as a result of their knowledge of 
the special populations. 



Another participant from a different state 
explained that the needs of students with diverse 
backgrounds were now required to be addressed 
in the curriculum and supporting instructional 
materials: 



Increasing alignment between State Content 
Standards and District Instructional Practices 

The extent to which the state directed or ap- 
proved district curriculum and instructional materi- 
als and checked that they were aligned with the state 
assessment depended on the philosophy dominant 
in the state: state control or local control. For ex- 
ample, in one state a participant explained that the 
actual curriculum taught in the schools was the 
responsibility of the local district and not the state. 
The state provided the content framework, learning 
standards, and practical examples of performance 
indicators, but would not recommend text books or 
tell local districts what they should be teaching: 

What we’ve been doing for the last decade is re-set- 
ting our standards. There are 28 standards and each 
one of these has what we call performance indica- 
tors, which are the classroom level indicators for 
teachers to pay attention to. These weren’t enough, 
people were saying what does this mean, what does 
this look like, help me find it and see it in my class. 

We don’t endorse textbooks and we don’t put out 
lists of publishers who have aligned their curricu- 
lum to ours. Where we draw the line is developing 
actual state curriculum— that we are not going to 
do unless we are forced. We’re still pushing back 
saying no this is your responsibility. 

Another participant from a different state ex- 
plained that the state content standards were aligned 
with state assessments. The state specified learning 
goals in the content areas that were assessed and 
had “back mapped” the content standards through 
the grades: 



I think the key thing here is that the needs of diverse 
learners are not an afterthought and not separate 
from the regular curriculum. It has to be interfaced. 
They are to be part of the entire curriculum through 
the day and then there’s additional support for them 
interfaced with the curriculum. 



Our content standards are integrated with our as- 
sessments for the state’s learning outcomes. We ac- 
tually developed a model for the standards in those 
content areas, we have worked with the content ar- 
eas to backward map, basically from high school, 
to 7 th , to 6 th all the way down to pre-K. Now these 
aren’t scope and sequence over the course of the 
school year, but by the end of the school year here 
are what the students should be able to do. 
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We asked this participant if the state planned to 
establish a state curriculum at any time and what 
the response of the school systems may be. This 
participant responded that a state curriculum was 
a strong possibility in the future and that districts 
were likely to have mixed responses: 



The visionary panel, I am pretty sure is going to 
recommend a state curriculum, how the school sys- 
tems respond is still... I used to say I could predict 
that, but no longer. I can identify school systems 
that are large, medium and small who want it and 
others who say this is the craziest thing, why are 
you doing this? 



not had the effect of uniting the educational profes- 
sion. 

Remember that when our standards were created, 
they were highly controversial, and they represent- 
ed the victory of one educational political pole over 
those who lost out. So there isn’t an educational 
community embracing the standards. The stan- 
dards turned out not to be a consensus document, 
but rather an agenda document - an agenda of a 
subset of educationally concerned people. We’ve 
got the standards, we’ve got assessments aligned to 
the standards, we have textbooks which represent 
an ideology with certain prescribed instructional 
strategies, not choice. 



A participant from a different state pointed out 
that although his/her state standards were voluntary, 
the state’s new assessment instrument and high 
stakes accountability system for schools was based 
on those standards and this obviously encouraged 
districts and schools to adopt them. Interestingly, 
interviews in this state took place in January 2002 
and this participant clearly identified NCLBA as a 
major force. 



The unique thing about this state is that the stan- 
dards are voluntary so we don’t have a mandate. 
However, the assessment system is based on the 
standards. But see I actually think the opportunity 
to learn is being built throughout the system and this 
is a very concrete example of it. The bigger picture 
is with No Child Left Behind and a standards-based 
model and for the most part many districts moving 
to standards-based report cards. If you are expected 
at the classroom level and the student level to teach 
the standards then there really has to be an oppor- 
tunity to learn. 



Interestingly, a different participant from this 
state had an alternate perspective on the effect of 
standards based reform. This participant believed 
that the state’s approach represented a victory of 
one educational philosophy over another and had 



One of EPRRI’s study states originally adopted 
a state curriculum over twenty years ago and has 
gone through several versions of state assessments. 
The state curriculum was made more rigorous in 
1997 and at the time of these interviews was cur- 
rently in middle of aligning its new assessments to 
the 1997 curriculum. This participant explained: 



The curriculum itself is not changing. It’s been in 
place for several years now and is the curriculum 
for all students, including students with disabilities. 
Our theory is that the curriculum ought to be in 
place for a generation of kids. We need to be care- 
ful because the new test will be based on the cur- 
riculum in place the day the test was made. 



We asked all participants for their perspectives 
on whether, at the classroom level, teachers were 
teaching to the state content standards. State infor- 
mants recognized that this issue was very important, 
but differed in their beliefs as to whether classroom 
instruction was aligned with the state standards. One 
participant explained that the skills tested at the 
high school level began back in middle school and 
that if the curriculum was not aligned then students 
would be in difficulty at the high school level: “If 
you don’t have aligned curriculum and textbooks, 
how far back does that set you? Because the math 
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standards — a lot of them come from sixth, seventh, 
and eighth grade.” 

A participant from the same state said that at the 
state level everyone understood the importance of 
the standards, but whether the districts and schools 
were implementing the state’s vision was uncertain 
and dependent on local leadership and the knowl- 
edge of individual teachers: 

The message needs to be strong and clear to the 
teachers and to the field in general that the depth 
and breadth of the standards should be taught. So 
I think the beauty of it is that finally, you know, 
everybody is on that same path and I think that for 
a state this large to finally get there is truly exciting. 
Now, how it’s applied is another story. I couldn’t 
guarantee you that every classroom is going to look 
exactly like the vision that everyone had, it will 
depend on the expertise of the leadership and the 
teachers and the professional development and all 
of that. 

A participant from a different state commented 
that the state had a lot of work to do with teach- 
ers in explaining what the state content standards 
were and how they could be taught at the classroom 
level: “So do I think it’s happening at the classroom 
level? I want to believe it is, but I think we have so 
much more to do.” This participant also pointed out 
that there was some resistance in the state from the 
teachers and the teaching unions about the content 
standards and that there was resistance because 
teachers did not really understand what they were 
to do: 

Here’s a story. Just a few months ago the state 
united teachers union asked us for a meeting to talk 
about the new curriculum and standards and they 
brought in teachers from school districts to talk 
with us about why we should stop. We asked the 
teachers how many had had their schools give them 
an opportunity to sit down and really be there to 
align what they had been doing to the standards. 
None had. So that’s why I say we have a lot of work 



to do. That’s where the rubber hits the road. Until 
we give teachers that opportunity to really sit down 
and see what it means and what it is going to look 
like for me, what are they to do other than say ‘this 
isn’t for me, it’s going to harm kids.” It’s just not 
real. 

We asked our state informants what the main 
barrier to curricular alignment was for students 
receiving special education services. Several in- 
formants across all study sites identified the lack of 
content knowledge possessed by special education 
teachers as the most crucial issue. For example, 
this informant explained that the special education 
faculty in many schools was not trained to teach 
content: 

The barrier is special education folks not know- 
ing what the curriculum is. But I don’t think that’s 
unusual. But you also have a lot of special educa- 
tion faculty and faculty in general who have been 
in schools a long time now and their training was 
not such that they were supposed to be able to teach 
math at a certain level or science at a certain level. 

So you have whole system that doesn’t believe in 
itself and so all that creates resistance to change, so 
you are going to come in and try to realign the cur- 
riculum with people that don’t even want you to do 
it. That’s a significant barrier 

Improving Educator Capacity 

It came as little surprise that the issue of edu- 
cator capacity was very much on the minds of all 
state informants. In many ways the data reveal little 
that is new on this issue, but rather underscore the 
pivotal importance of teacher quality to student 
learning. When we asked how schools could im- 
prove the performance of low performing students 
and schools one participant said: 

My first thing is, we’ve got to have the best teach- 
ers that we can possibly have. To move those kids 
around the first critical thing is the quality of the 
teacher. So our lowest performing schools in the 
state are typically ones where we have the least 
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experienced teachers and the highest turnover of 
teachers. So I think that the first thing is stabilizing 
and upgrading the quality of the teaching. If you 
don’t have a good teacher there’s no curriculum 
that’s going to meet the needs of all those kids. The 
teacher is the person who sits down with that group 
of kids, the teacher needs to know these are the skill 
sets and then they need tools to help the students 
get there. 

Another participant from a different state agreed 
and added that two issues were involved. The first 
issue was the preparation of new teachers and the 
second was to improve the capacity of existing 
teachers and principals. This participant believed 
that the latter area was neglected in the past: 

There has over time been a realization that if you’re 
going to increase student achievement you need to 
start looking at the quality of the teachers. We have 
two issues going on at the same time. One is try- 
ing to ratchet up the qualifications and preparation 
of teachers. The other issue that has been a little 
slower to progress, but is increasingly getting the 
attention of our state board and our state superin- 
tendent, is the capacity of the existing teachers and 
elevating the capacity of teachers - not only teach- 
ers, but principals - in terms of focusing on instruc- 
tional outcomes. 

Improving the quality of new teachers. All of 

EPRRI’s study states were aware of the threat that 
a lack of teacher capacity posed for the success of 
accountability reform and all had initiated policies 
to respond to the problem. States adopted several 
approaches such as increasing the rigor of licens- 
ing examinations, revising teacher standards, and 
redesigning teacher preparation programs. 

Several states have made changes to the teacher 
licensing examinations in light of the belief that they 
lacked rigor. One participant described the state’s 
teacher initiatives as part of the overall plan for 
increasing student achievement. The new examina- 



tion requires initial certification teachers to pass the 
content specialty test. 

In 1996 the Chancellor convened a task force on 
teaching to try to define what kind of teachers we 
want in our system to help ensure that all students 
moved to the higher learning standards. We now re- 
quire in 2004 preservice teachers to pass the liberal 
arts and science test, the assessment teaching skill 
written examination, and the content specialty test. 

A participant from a different state explained 
why the state changed from one teaching ex- 
amination to another. In addition, this participant 
explained that it was important to study the rigor 
of examinations and passing scores overtime to 
maintain the quality of the teaching force: 

We have had teacher prep tests in the state since 
1987 and in 1987 adopted the National Teachers’ 
exam. What we were finding over time was that it 
didn’t take very long for the scores that we set in 
87 to be less than rigorous and that our pass rates 
were really in the 96-97%. So over time what we 
learned was that it’s important to look at and revisit 
our teacher certification test, so in 1998 Praxis I and 
II were brought on board. 

One participant described the role that special 
education had played in the development of new 
teacher standards: 

We helped in part of the review of the standards 
for the new special education credentials and re- 
ally fought hard to make sure that they had reading, 
math, and behavior embedded in the standards. So 
that whoever came out of a school with a special 
ed. degree could teach reading, could teach math 
and could manage behavior in an effective way. 

However, a participant from a different state 
commented that the division of special education 
had not been involved in determining the knowl- 
edge and skills that special educators should have in 
that state. This participant described how the teacher 
credentialing board had a monopoly on policy and 
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that as result the division of special education had 
little influence over teacher standards for special 
educators. This participant felt that the quality of 
teachers had suffered because of this: 



Nobody is at the table for teacher credentialing, ex- 
cept the commission on teacher credentialing. They 
are a powerful entity unto themselves and they’ve 
mucked around with credentials and college re- 
quirements and we’re getting very poor quality 
teachers. They’re very busy putting together the 
graduation requirements and the testing require- 
ments for all that stuff. But they’re not looking at 
turning the university system on it’s ear, which is 
what they have to do. 



All of EPRRI’s study states reported that they 
had state-level initiatives to improve the quality 
of teachers by establishing more rigorous teacher 
standards for preparation programs: 



The teacher education standards that we use to reg- 
ister teacher education programs ensure that all pro- 
spective teachers are able to teach the state learning 
standards. You look at the learning standards books 
and they’re rather comprehensive in terms of all the 
things that kids have to learn from k-12. 



An informant from a different state explained 
that the state now required preparation programs to 
include rigorous academic preparation and longer 
field experiences. This participant also discussed 
the pivotal role of federal legislation in enabling 
states to influence the content and design of teacher 
preparation programs: 



Title II is probably the biggest effort thus far to put 
some accountability on teacher prep programs, and 
so that is underway in our state. Title II requires 
for the first time teacher prep programs to report 
the pass rates of their program completers on the 
teacher certification tests and in our state we’ve had 
some schools that have had low pass rates. I have 
personally met with the presidents of each one of 
those campuses because we feel it’s important for 



them to know, because the stick is that these cam- 
puses will have to report in all of their publications 
that they are a low performing campus. 



The same informant emphasized the crucial role 
that higher education must play in improving the 
quality of teachers and pointed out that higher edu- 
cation needs to be part of the information loop: 



Teacher prep is a very important aspect of higher 
education’s contribution to the quality of our K-12 
public schools. There needs to be a strong connec- 
tion between teacher preparation programs and K- 
12 priorities in terms of what the state and federal 
government are asking of teachers and of students 
so that the teacher prep programs wouldn’t be 
working in isolation of what K-12 schools are being 
asked to do. And also in keeping with the redesign 
and understanding K-12 priorities, they need to un- 
derstand that we have a very rigorous state assess- 
ment program and it has, over the time, provided a 
tremendous wealth of trend information. 



An informant from another state pointed out that 
the state hoped to improve the skills of both regular 
and special educators through changes in teacher 
preparation requirements. In terms of teacher qual- 
ity, this participant believed that a hybrid teacher 
would be the next natural step in teacher prepara- 
tion, especially at the elementary level and echoed 
the need for strong content preparation for special 
educators: 



We have a sort of strategic overall goal, which 
obviously has to do with teacher capacity. It’s re- 
ally personnel capacity and we’re working on it at 
different levels. For instance you have the recent 
change in teacher preparation programs and that’s 
aimed at preparing a much more content strong 
special education faculty. There will be more em- 
phasis on the general education side on disability. 
So long as there is a general education credential 
and a special education credential they are at least 
to be more conversant with each other. A lot of our 
higher education institutions, however, go into a 
sort of blended programming, I guess you’d call it. 
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We asked participants the extent to which they 
perceived that higher education was responsive to 
the needs of schools and districts. Several partici- 
pants perceived that higher education needed to be 
cognizant of what teachers were actually required 
to do in the classroom. One participant described 
this as a key component of the state’s redesign of 
teacher education: 



One of the theoretical underpinnings of this is that 
teachers in K-12 schools would benefit from work- 
ing with higher education faculty who are on the 
leading edge, the cutting edge of teacher prepara- 
tion. And that also higher education faculty would 
understand the day to day operations of schools and 
tailor and reengineer their teacher preparation pro- 
grams to take into account what teachers are really 
being asked to do. 



Another participant from a different state ex- 
plained that college presidents were unhappy at 
having to spend money to improve programs, but 
that the Deans of education were more positive 
as the state’s new requirements meant they could 
argue for the need for more resources in education. 
However, the smaller independent colleges with 
less money found it harder to respond to increased 
state requirements: 



The college presidents bemoaned anything that 
cost more money, whereas the Deans welcomed the 
ability to leverage greater resources out of their in- 
stitutions. One large institution knew that they had 
to do a better job, they absolutely knew and they 
brought in a university dean and revamped the pro- 
grams. The commission of colleges of independent 
colleges, they were split. The bigger ones didn’t 
have a problem, the smaller ones did because of the 
financial strain. The independents thought it was 
very much the role of the board of trustees at the 
individual institutions, and it’s a very interesting 
issue. We are requiring all our teacher education 
programs to be accredited by a teacher education 
accrediting body. 



The same informant discussed the reaction 
of university faculty to the state’s initiatives to 
improve teacher preparation programs. From the 
perspective of this participant, faculty viewed 
the state’s restriction on semester teaching hours 
unfavorably. 

Faculty is tough. Because we put restrictions on the 
number of semester hours you can teach a semester 
in education. The intent was to break away from 
the cash cow approach where we just funnel as 
many kids as we can through the school of educa- 
tion and make money off them. So we said, no, this 
is an academic discipline, you have to only teach x 
numbers of semester hours per faculty person, that 
caused a lot of stink. 

A participant from the same state confirmed the 
initiative and commented that to improve the qual- 
ity of teachers the state had to work on the teacher 
education faculty first: 

Here’s my problem. Who is going to be teaching a 
teacher, and who is going to tend to that? That’s my 
concern. Who’s going to train the professor. Our 
idea this year is going to be developing materials to 
train the professor on how to teach the content, the 
pedagogy, and the key ideas. 

Another participant in a different state described 
an initiative directed toward university professors 
who teach elementary and special education: “Our 
special ed. professors at all universities are being 
trained in the very best reading research based prac- 
tices. So that when they train their students these 
kids leave university with the stuff they need.” 

In attempting to solve the teacher shortage 
crisis, several states had allowed alternate certifi- 
cation programs to be established. One participant 
explained that it was very important to the success 
of the alternate route that institutions of higher edu- 
cation be involved so that stakeholders viewed them 
as comparable in quality to traditional programs: 
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I was one of the architects of the alternative certifi- 
cation program, and we very precisely, very clearly 
left it within the realm of higher education because 
we knew that we needed to keep them engaged and 
we knew that we had to demonstrate to a larger 
community that these are not second rate teachers, 
they’re going to have the same qualities, they’re 
just going to have a different approach. 



A participant from the same state described the 
approach taken toward second career individuals 
and emphasized that the policy was designed to 
attract successful individuals who could bring a 
different approach to teaching. 



We created a new certificate called a transitional 
certificate to try to bring second career individuals 
into the teaching profession. Our intent is to take 
people who are trained, that have made it in another 
career that are interested in changing. They’re not 
kids, they’re adults and they bring a different set 
of skills to the table than a 21 year old entering a 
tough school. That’s our hope of trying to widen the 
net in terms of capturing more people in the teach- 
ing profession. 



However, participant perceptions of the likely 
success of this approach were mixed. One par- 
ticipant from another state expressed doubts over 
alternate certification programs directed to second 
career individuals. This participant did not think 
that second career individuals made good teachers 
and that the numbers of second career individuals 
were simply not great enough to make a dent in the 
teacher shortage problem: 



I’m not a big fan of trying to recruit second career 
folks. I’ve not seen that that produces a big number. 
I can take you to a number of folks who have come 
into teaching from those careers and suddenly are 
confronted with a bunch of kids. They say ‘Now I 
know why I didn’t have kids’ and ‘now I know why 
I’m glad my kids left home’. And so they certainly 
can’t cope with the energy, the diversity, all those 
issues. 



An informant from a different state commented 
that the quality varies for alternative route candi- 
dates 

In terms of pass rates, the alternative programs have 
varying rates of success. Some are quite high, some 
not so high. I don’t know what will happen with 
the new tests in 2002. Some people predict well be 
losing teachers because the tests will be harder and 
they encompass more content areas. 

The data revealed that states had identified 
several innovative initiatives to try to attract new 
teachers to the profession and to try to retain newly 
qualified teachers. One participant described several 
state office recruitment strategies at the college and 
high school level: 

First, they are trying to use the money to free up 
professors so they could spend more time recruiting 
potential students much the way we recruit foot- 
ball players. Where the professor gets to know not 
just the kid, but the parents. The second part, the 
districts offer to kids at the high school level the 
elective Peer Assist for Students with Disabilities. 

One university program has done a data run on the 
kids that took that course and then went to them and 
offered summer camps around the state and asked 
would you like to be a special educator? 

Another participant from a different state de- 
scribed other initiatives directed toward increasing 
the number of teachers in the state: 

We are trying to get other teachers from other states 
in here and give them a couple of years to pass the 
test. We have legislative proposals trying to allow 
teachers who are retired to come back in without 
pension penalties and they could teach part-time. 

We have legislative proposals on pension portabil- 
ity among teachers. 

In another state an informant described some 
district level initiatives in which local districts ad- 
opted a “grow your own” approach in partnership 
with smaller universities: “A lot of school districts, 
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in particularly large school districts, build relation- 
ships with their neighboring universities and they’re 
looking at creative ways to encourage people to 
come into teaching and to stay in teaching.” 

The data revealed that participants regarded the 
issue of retention as a major problem, especially 
in the early years of an individual’s career. For 
example, one participant commented: 

The data are showing that we’re getting more and 
more first year teachers into the workforce than 
ever before. Whereas 10 years ago about 38 per- 
cent were first year teachers, and it’s up to like in 
the big cities 65 percent of the new hires are first 
year teachers. 

Although several initiatives to support begin- 
ning teachers were described by participants, it was 
clear that funding for these initiatives remained 
uncertain in the long term. One participant de- 
scribed his state’s beginning teacher support system, 
which was a pilot program serving 10 percent of 
the state’s new teachers. This participant explained 
that the program had funding problems and may 
be abandoned if a new funding source could not 
be found: 

The state is experiencing a teacher supply problem 
in the areas of special education, foreign language, 
math and science. It is at a critically low level. 

It’s more a problem of retention than supply. Fifty 
percent of teachers here leave the profession after 
five years; one third leave after three. It is a com- 
mon problem in most states. Right now we have 
10 percent of teachers being supported by a be- 
ginning educator support system. It would need to 
be available to approximately 14,000-17,000 new 
teachers in the state each year. Federal funding for 
this program disappears at the end of this year. We 
are working now to get more money and if we can 
get to where we’re supporting even half of the new 
teachers we can keep teachers in the classroom. 



A participant from a different state expressed 
similar funding concerns as to whether the legisla- 
ture would provide the resources the SEA had asked 
for to support the first year mentor program. “The 
first year mentoring: will the legislature support that 
with the dollars we’ve asked for. It’s not so much a 
new policy, but whether or not this existing policy 
has to be adjusted.” 

Participants also pointed out that new teach- 
ers may not have the pedagogical skills that they 
needed to teach children with a range of abilities. 
For example one participant explained that teachers 
need a variety of instructional strategies to teach the 
wide range of abilities found in regular classrooms: 
“It’s critical that we provide preservice teachers 
with real world and intensive training so that they 
can serve any child in the regular classroom and 
not worry if they have a disability or not. That’s 
number one.” Another participant from a different 
state explained the situation in more detail, saying 
that having content knowledge was just one part 
of being a teacher. This participant believed that 
a teacher needed to know how to teach different 
types of students too: 

Many of the skills that we want teachers to possess 
are beyond academic areas. Part of what we want to 
do is make sure that all students are successful and 
all teachers must be prepared to teach all students. 

So there is a big component of inclusion— teach- 
ers have to be prepared to teach students with spe- 
cial needs, and they have to be prepared to teach 
students of low socio-economic background, they 
have to be prepared to teach kids who are English 
language learners, they have to be prepared to teach 
students of color. 

An informant from a different state commented 
on the need for teachers to learn different strategies 
for teaching students with different learning styles. 
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From this participant’s viewpoint there are a large 
number of students with problems who could be 
taught if teachers have the skills they need: 

We have a lot of kids with a lot of problems. I first 
started in general education; if you learned good 
teaching you learned good teaching. You learned 
straight-faced and focused on the child. You looked 
at developmental levels and if you focused on the 
interaction of the kid and meaningful activities and 
you make it all make sense. You don’t need special 
education. Yes I acknowledge there are tremendous 
variations in learning styles across kids and it’s 
a problem because you do need to have different 
tricks in your little bag to pull out and try to work 
with different kids. We’ve got so many kids who 
have problems it’s the norm now, it’s not a disabil- 
ity, it’s a normal range of individual differences that 
take teaching skills. 

On the same subject, this participant from an- 
other state explained that increasing the ability of 
new teachers to work with diverse learners was the 
third component of the state’s overall strategic plan: 
“A third component is exposing teacher candidates 
to a diverse student population. And diversity is 
broad in the sense that it does include special edu- 
cation. But diversity includes students of varying 
socio-economic strata and ethnicity.” 

One participant pointed out the conflict between 
teacher quantity and teacher quality at the policy 
level. Schools and districts need to have teachers 
in classrooms and therefore need to get prospective 
teachers through the certification system as quickly 
as possible. On the other hand schools also need 
teachers who are competent-both in content and 
pedagogy — a task that is not necessarily amenable 
to cutting comers: 

When they did this whole plan, they understood 
that they had two main objectives and they were 
not necessarily compatible: quality and quantity. 

You don’t want to short-change kids with teachers 



that are inadequately prepared. At the same time we 
have to face the fact there’s a supply and demand is- 
sue, so we are going to have to look at everything. 

These sentiments were echoed in other states. 
For example, one participant pointed out that just 
as the state was trying to raise teacher standards, 
fewer people wanted to become teachers and that 
this was working against them: 

We’re facing a demographic problem. We’re trying 
to raise the standards for both the content knowl- 
edge and the pedagogic knowledge of our teach- 
ers, at the same time that the number of people 
who want to be teachers is not growing or at least 
not fast enough to solve our teaching problems. 

Or teachers who are maybe willing to go to some 
schools. That’s a complex issue. It starts with your 
best teachers and your best principals. Now how do 
you get them there? 

Increasing the capacity of existing teachers 
and school leaders. 

Data from across all study sites revealed that 
informants were very concerned about the needs of 
existing teachers and personnel shortages. However, 
in three out of four states the informants commented 
that higher education and the districts needed to 
address the issue of professional development. For 
example, this participant described the complexity 
of the issues, but reiterated that existing teachers 
needed support: 

We import a lot of teachers so that’s why you can’t 
just tackle it on the front end, because as teachers 
come in either with experience from other states 
or from other states with no experience, there’s an 
in-service or professional development component 
here that needs to be addressed and picked up by 
the district and higher education has a role to play 
in that. I mean, they can be a provider of quality 
professional development just as anybody else can. 

But attention needs to be focused on the incumbent 
teacher. 
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A participant from another state pointed out 
that the state laid down guidelines for professional 
development, but that within those parameters the 
local district, teachers and administrators had to 
come together and determine their needs: 



or pedagogical content. Once they get their profes- 
sional certificate, then instead of that being a per- 
manent certificate they get to keep forever and ever 
they have to keep that professional certificate alive 
by completing 175 hours of professional develop- 
ment every five years. 



The concept was to decentralize as much as you 
can. There’s no fountain of wisdom from the SEA. 
All the research that we looked at showed that 
when professional development was locally devel- 
oped and was focused on what kids needed to learn 
and when all the other noise went away it had a 
measurable impact. Districts have to do an annual 
plan, a professional development plan based upon 
two key variables: teacher capacity and student 
learning needs. It has to be decentralized, it has to 
be done locally, by teachers and administrators, and 
approved by the local board of education. 



Another participant from the same state ex- 
plained that as a result of accountability reform the 
state now instructed districts how to spend their 
state professional development funds: 



So now we look at the data and say that 80 percent 
of the money we give you this year is going to have 
to focus on these things in this region, based on test 
scores and the other key indicators. Then you can 
use 20 percent of your money to your own local fla- 
vor. Now they then go to the schools and look at the 
data. It’s not so open ended anymore. The things 
that people are interested in going into staff devel- 
opment are very different than they used to be. 



A participant from a different state described a 
similar initiative: “It changed from a lifetime cer- 
tification to a 5 year renewable certification with 
150 hours of professional development education 
required. Teachers are required to keep records 
of professional development hours, which aren’t 
checked unless there is a problem.” 

A different informant from this state commented 
that sometimes districts were unwilling to give 
teachers release time to attend training courses at 
the educational service centers. In an attempt to 
solve this, the state legislature will require districts 
to provide training in special education directed 
toward regular education teachers: 

It’s still up to the district to release its personnel 
to the center to receive training and we hear a lot 
of complaints from teachers that their principal or 
superintendent won’t let them go. A new state law 
was passed just this spring that for the first time 
beginning in the 2002-2003 school year requires 
districts to provide annual training in special edu- 
cation and target it specifically to general education 
teachers. 



The same participant went on to explain when 
the state altered the existing certification system, it 
also revised professional development requirements 
to ensure that from 2004 onwards, teachers who 
entered the profession were required to take 175 
hours of professional development every 5 years in 
order to remain certified: 



These new teachers that are coming in 2004, they 
will get an initial certificate, after 3 years they get 
their masters, which has to have 12 hours of content 



One participant from another state expressed 
frustration at what he perceived was a negative 
attitude of school boards of education toward pro- 
fessional development: 



The key is, and always will be and never will change, 
is showing the boards of education there’s value in 
this activity. Until a superintendent does that, then 
forget it. Many people consider professional devel- 
opment a sop to the unions so that teachers get the 
time off to do whatever they want to do. The mind- 
set, I mean, when you talk about a 12 billion dollar 
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industry and continued development of your work 
force is not part of it you wouldn’t be a CEO for 
very long. But education works differently, there’s 
a different set of priorities and people controlling 
it. 

The data revealed that building the capacity of 
the teaching profession to teach mathematics and 
reading were the focus of state resources toward 
staff development. Several participants pointed out 
that many teachers, especially at the elementary 
level, may not know the content standards that 
they are expected to teach and may not have the 
content knowledge to teach those standards in their 
classrooms: 

They are not seeing the significance of taking those 
standards and bringing them down to the classroom 
level into instructional strategies for youngsters to 
learn and for teachers to know. How many teachers 
know how to teach algebra to youngsters at the 5 th 
and 6 th grade level? They don’t have that content 
background. The teachers are saying to me they 
don’t have the content background or they don’t 
have the instructional strategies to do this and I 
blame myself. I say what am I going to do to make 
sure that we build that for them because I have to 
say part of those scores are a direct link to the stan- 
dards in my lap. 

Another participant from the same state com- 
mented that special education teachers in particular 
need more math content area knowledge: 

Most of our special education teachers are in ele- 
mentary buildings, but practically speaking, we’ve 
talked to a lot of special education teachers, it’s 
math. What it comes down to is better preparation 
in mathematics for a lot of these special education 
teachers who have to deal with that resource room. 

Another participant from a different state echoed 
this perspective: 

In this state probably 65 percent of the middle 
school math teachers are elementary certified. I 
would argue that if you are elementary certified that 



almost guarantees that the last math you took was 
in middle school. You probably took the low level 
high school math; maybe you want to go to college 
so you struggle through an algebra and geometry, 
but you didn’t excel. Then you went to college and 
you got the same math repeated. Now you come 
here and I’m going to ask you to teach box and 
whisker plots. 



We asked what strategies the states were using 
to improve professional development. Participants 
described a number of research-based initiatives 
for improving educator ability to teach mathemat- 
ics and reading:. 



Programs are research based and needs driven for 
math teachers. Master trainers are provided to each 
of the 20 regions of state programs. For example: 
a 5 day, 40 hour training in Algebra I. New math 
courses include teaching teachers how to work with 
manipulatives and different learning styles. 



Another participant from a different state com- 
mented, “We have what’s called a reading and math 
initiative and we hired people to focus on staff de- 
velopment in reading and math. They are making 
sure that the stuff being used is soundly researched.” 
Yet another informant from this state told us: “All 
of the professional development has to be focused 
around reading and improving our teachers’ skills 
in that area.” Finally, this participant from the 
same state described a reading initiative directed 
toward teachers in the early grades. The goal of 
the initiative was to reduce the number of students 
who entered special education in 3 rd grade because 
they could not read, by giving teachers the skills 
to teach reading: “The reading initiative for K-3 is 
huge. We train every teacher in the state and we 
pay them a stipend to come to training. One hope 
is that the reading initiative will lower the number 
of children coming into special education at third 
grade because they don’t know how to read.” 
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The issue of adequate resources to provide the 
professional development was discussed in several 
study states. One participant commented “I think 
we are struggling with how to apply this and with 
the resources that you have, how to make it work?” 
Another participant from the same state explained 
that: 

Let me back up a minute, I have to tell you what 
our staff is. We have one math associate in our of- 
fice. We used to have a bureau of math associates, 
8-10, now there is one. So in our thinking this year 
of how we are going to leverage our resources and 
make sure that we can really have an impact, what 
can we do with one? 

Participants from across all four study sites 
perceived the leadership skills of district and school 
administrators to be of crucial importance to the 
success of educational reform. One participant 
explained: “It’s amazing how much leadership 
makes a difference, it really is. And that’s why 
you can find these schools all over the state and 
all over the country that shouldn’t be doing very 
well, but are.” 

Another participant commented: 

My sense is that there are districts that haven’t got- 
ten any better even with state monitoring because 
they lack leadership. What we are finding out in the 
state is that no matter what else you do, you need 
at least competent leadership, and if you don’t have 
that you are going to be struggling with those dis- 
tricts for a long time, because they can’t respond 
without it. 

The same participant explained that if a key leader 
left, then the district can quickly have problems: 

One of the things we have found is that you could 
have a district that is doing well and the director of 
special education and the superintendent leave, and 
the next year they start to have problems because 



a new person comes in and doesn’t have the same 
level of leadership skills. 

Of real concern to all participants was the short- 
age of school and district leaders. This participant 
pointed out that the demands placed on school 
leaders were increasing and to meet them school 
leaders needed to be of high caliber: 

Who are the next generation of leaders because that 
is going to be a major problem? School and dis- 
trict leaders; and then they’re also going to have 
to be the brightest and the best because there is so 
much more expected of them. Which is one of the 
reasons why we think there will be a higher turn- 
over of leadership soon, or is now, I guess, or hard 
time replacing them is another way of putting it. 
Because some of the people aren’t up to the task 
and the task is much more difficult than it ever was 
and you’ve got to be really good and thick-skinned 
to be an educational leader today. 

Another participant explained that the ability of 
the school principal to be an instructional leader was 
crucial in education reform. According to this par- 
ticipant the school principal needs to be responsible 
for curriculum alignment with standards, hiring 
teachers based on the school needs, and leveraging 
the resources needed at the school level: 

The principal is the instructional leader. So if the 
principal doesn’t know the kind of performance ex- 
pected of his or her staff then it’s a giant gamble 
that the staff is going to know. If you’re in a middle 
school and you’ve got 20 math teachers and they 
are all teaching 20 different targets the school sys- 
tem may have a curriculum in place, but the imple- 
mentation is really at the principal level, the school 
level. Now it has to be the principal identifying 
what a teacher can and cannot do, and the principal 
working with that teacher saying this is the training 
you need, and the principal working with human 
resources saying this is the kind of teacher 1 need 
coming into my school, and these are the resources 
I need to train that teacher, and this is how I am 
supporting the growth of that teacher. You need a 
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great principal who is the instructional leader, not 
the building manager. 

A participant from a different state reiterated 
the seriousness of the leadership issue at the school 
campus level and provided a picture of the stresses 
that principals faced on a daily basis. 

It is absolutely going to be as significant a prob- 
lem long term because of the climate and it’s not 
fun being a school principal. It’s conflict with the 
boards of education, it’s the remuneration level, it’s 
the stress level, it’s the leadership of the academic 
learning, student learning, it’s all these different 
jobs, plus now we have the whole thing about you 
have to do disaster preparedness, sexual predators, 
all these different things that you have to be atten- 
tive to as a school leader. It’s not like it used to be. 

Another participant from the same state ex- 
plained how principals reacted when she suggested 
that they talk with their faculty about the standards 
and the curriculum. She commented that they felt 
swamped by other issues: 

But given all the pressure, look at our leaders, all 
of the pressure is on them right now. And I asked 
them why can’t we have that conversation [on the 
alignment of curriculum and instruction] at faculty 
meetings? They said it’s complicated, we have a 
million other things to worry about. I want to say 
wipe them off the table, here’s where we are, this 
is what you need to focus on. As educators this is 
the curriculum, this is what parents want to know: 
What are you doing and what are you teaching? But 
I am not in their shoes and I believe there are so 
many other competing forces here that this is going 
to take time 

Aligning Special Education Reform with 
General Education Reform 

State informants discussed two aspects of align- 
ment of education reform. First, data revealed that 
state special education personnel were in the midst 
of changing their monitoring of special education 
because the U.S. Department of Education’s Of- 



fice of Special Education Programs (OSEP) had 
adopted a continuous improvement model for states 
and also because of changes in IDEA’ 97. Second, 
they were concerned with how reform in general 
education would align with more traditional special 
education practices. 

We asked state informants what their experi- 
ences were with OSEP’s new monitoring approach. 
One participant argued that OSEP should take a 
more problem solving approach and not an identi- 
fication approach: 

I went to OSEP and I said this is crazy, this whole 
system is nuts. You come in every 3 or 4 years and 
you tell us what we have already told you and then 
you write a report as if it’s all new. You fund all 
this research, all this technical systems and sup- 
port research efforts all across the country. Yet you 
don’t ever make an effort to come here where we 
are open to assistance and try to put your resources 
to bear on problems that we already have. So they 
agreed. We’ve identified problems, we’ve verified 
the problems and now they’re going to help us. 
There is just no way that they can know our prob- 
lems like we do. But there are ways that they could 
help us that we don’t know of. 

In general participants were very positive about 
the new federal approach to monitoring. One par- 
ticipant said: 

One of the things I really like about it is, not only 
having all the stakeholders involved gives a differ- 
ent perspective on how people are seeing things, 
but because it is data driven, it has really given us 
an eye-opening experience in looking at areas we 
really need to work on. We have a lot of data, we 
collect a lot of data, but what do we actually do 
with it? It’s been a big issue, so now this gives us 
an opportunity to take the data and focus it and use 
it to make better decisions on what we do for kids 
as a whole. So I really like it. 

Another participant from the same state added 
that everyone hoped that at OSEP’s next visit they 
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would recognize the continuous progress that the 
state had made on indicators. This participant 
thought that people may become frustrated if the 
progress was not recognized: 

I think everybody from the agency involved is pret- 
ty positive, but is concerned that if OSEP comes 
in April/May, visits seven districts, interviews 12 
people and then issues findings that the state is de- 
ficient in LRE etc. If OSEP doesn’t honor the work 
we’ve done then people will feel very frustrated. If 
they say “you are making progress on this, we’re 
satisfied that you are aware of your own problems”. 

If they work with us like that 1 think it would be 
very positive. 

Likewise another participant from a different state 
commented: 

That’s the struggle with OSEP. I’m hoping they are 
talking about focused monitoring like they mean it 
because I would like to be able to parallel that in a 
certain way. But it’s yes and no. I mean they look 
at eligibility documents, they look at what you are 
doing, if you are missing two lines out a sentence it 
still can be a problem and I just think we don’t have 
enough time to concentrate on that. We have to look 
at what’s important and deal with that. 

Another participant described how the new 
CIMP monitoring approach occurred in practice: 

We got a huge constituency started with 70 some 
people, we have 2 meetings a year of our KPI 
stakeholder group, we’re coming up to that in Janu- 
ary. They represent general ed, special ed, all the 
special interest groups, universities, school boards, 
everything. And we developed goals, performance 
goals and indicators, we developed and prioritized 
the kinds of data elements that we collect now and 
developed a system of monitoring and outcome 
oriented activities for kids with disabilities so that 
we’re looking at improving outcomes rather than 
just compliance. 

State informants were very much in agreement 
that the traditional approach to accountability in 
special education of holding states and districts 



accountable for compliance with process had not 
resulted in improved outcomes for students. One 
informant had worked for a school district before 
moving to the state office of education and de- 
scribed a “gotcha” situation during which the state 
office personnel reviewed educational records and 
found instances of non-compliance: 



Basically, I can tell you from my district experi- 
ence. What happened was you got a report back 
from the state because there is only a record review 
so that was it. They would say out of this many re- 
cords, these number were out of compliance. And 
I’d be screaming and yelling, “well tell me which 
schools” and they’d say “No, here’s your data.” 
And finally I did get it by school and I could see it 
was one school that screwed up and they misinter- 
preted X or Y or Z and I could go in there and fix 
that. It was largely done by memo. The director of 
special education would say, “Ok gang. As a district 
we screwed up on— so make sure your notices are 
out within ten days.” I mean that was basically the 
route it took. There was no other accountability. 



A participant from another state explained the 
situation in similar terms, as one in which process 
was the dominant theme of special education moni- 
toring for accountability purposes: 



Somebody was talking about do we really have ac- 
countability. Well we do, it’s just not what we want. 
We probably have real high accountability for dot- 
ting “i’s” and crossing “t’s.” and process. We prob- 
ably don’t have the kind of accountability we want 
as it relates to student results. That change is going 
to be tough because a lot of people have defined 
themselves by the process. They become the rule, 
the form, or the transition person. Well should you 
really be the transition person or should you be the 
“I’m increasing graduation rates person.” 



We asked participants what impact process 
oriented accountability had on student outcomes 
for students with disabilities and if school practices 
improved as a result. Participants’ responses were 
very similar on this topic. For example, one partici- 
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pant explained how the process oriented approach 
took time and resources away from improving 
outcomes for students with disabilities: “In some 
cases it led to worse outcomes because that’s what 
you are going to spend your time and focus on. And 
it’s not real.” 

Another participant from a different state ex- 
pressed similar concerns and reflected upon the fact 
that the goal of teaching and learning can become 
lost in efforts to fulfill procedural requirements. 



Comments from some states reveal they had 
made changes monitoring activities in the districts. 
In particular, the focus on outcomes and results in 
IDEA 97 removed some of the “clutter” associated 
with special education accountability and allowed 
state personnel to help districts solve problems and 
not just identify the same problems repeatedly. One 
participant explained that as the department became 
more confident in quality of the data it could move 
toward solving the systemic problems and working 
with districts: 



I’ve been very frustrated over the last few years 
with the clutter. I understand one person’s clut- 
ter is another person’s right, but somewhere along 
the line, the process- and the goal- of teaching and 
learning gets lost in making sure your “i’s” are dot- 
ted and your “t’s” are crossed. Somewhere along 
the line the teacher and the skill sets the teacher 
has get neglected because you’re so worried about 
whether they’re filling out the forms. Look at the 
amount of time they are spending doing things that 
may not have anything to do with whether or not 
they’ve taught a kid. We have staff to go out and 
monitor and report compliance data. Now what did 
we do with it? We sent a paper saying swear to god 
never to do it again, then they would send a pa- 
per back saying we swear to god we’ll never do it 
again. Now what did we learn from that? 



Another participant from a different state de- 
scribed how he/she had stopped worrying as much 
about time lines although they were still a large 
part of his/her work. Instead this participant had 
started thinking more about indicators that led to 
student success: 



It really does come down to what indicators are 
most closely correlated with success and so what 
indicators do you focus on and why are they cor- 
related. I’m not worried about the 32 days even 
though right now I’m spending time counting the 
32 days. I think we can make a strong case for re- 
ally thinking differently about this because to me 
the purpose of quality assurance is to improve pro- 
grams. 



We are examining even the verification review, and 
this is just, this isn’t even official it’s just a discus- 
sion we’re having internally saying at this point 
after 5 or 6 years of data collection we have a lot 
more confidence in the accuracy of the data. We 
shouldn’t be spending all of our time verifying the 
data. If these are the issues that are coming up in 
60% of districts that we visit then we can be pretty 
sure that they are real. Do we have to keep identify- 
ing them, or can we start addressing them? We’re 
moving more toward collaborative reviews as be- 
ing kind of the focus in really the poorer perform- 
ing districts where we spend a lot more time. 



Another participant from a different state ex- 
plained how reform in special education had nudged 
special education and regular education closer to- 
gether and each was beginning to look at the same 
data at the same time: 



I think the monitoring really is going to start driv- 
ing more how the state does its business, at least 
the division of special education. First of all the 
new monitoring system in OSEP is data driven, 
but we’re moving toward a much more non-cycli- 
cal model, meaning what we’re looking to do is 
integrate it into routines. Special education has 
always been one step behind, or at least not in sync 
with the school system. When the school system 
looks at their exit data, looks at their graduation 
rate, does diploma versus certificate, that’s when 
special education should be looking at their data. 
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We asked the state-level participants whether 
there was resistance to outcomes based accountabil- 
ity reform in special education. Several participants 
explained that the change from process to results 
was hard for some districts, schools and teachers 
because it required a paradigm shift. For example, 
one participant explained: 

Do we measure effort which is easy or do you 
measure effect which is the hardest thing to mea- 
sure. But it’s the most important thing to measure. 
Counting kids that are served, that’s easy. Counting 
results for those kids is hard. But that’s what you 
really want, so we’re moving folks in that direction. 

I tell them that they must balance this thing, if you 
focus on teaching and learning, everything else will 
take care of itself. That’s a hard concept because 
when it’s all said and done, the process is actually 
the easiest thing to do. The hard part is teaching 
the kids reading, teaching the kids math, and get- 
ting along with the parents. That’s the hard stuff. 

The easy stuff really is doing the paperwork, even 
though everybody hates it, it’s easier than being re- 
sponsible for learning. That’s what makes a tough 
sell. Even though that’s what everybody wants, 
when push comes to shove, it’s more difficult. 

A participant from a different state commented 
that some district special education directors do not 
know how the outcome data to make decisions: 

There are some special ed. directors who are very 
proactive, they are taking their data, breaking it 
down on the campus, publishing it district wide so 
every campus knows what every other campus is 
doing. Those are situations we are seeing a lot. But 
we still have some that aren’t. 

Another participant from a different state com- 
mented that although in the future the districts 
would be able to use the information and, based 
on that, change course, the new special education 
accountability system was so new that the districts 
did not fully understand it: 



The good thing for the districts is that it’s a very 
progressive model and they can know as they’re 
getting worse and worse and hopefully they’ll take 
action to do otherwise. 1 think our challenge right 
now is getting them to understand the system, how 
this is working, and what does it mean to them? 

Participants from several different states per- 
ceived that districts were suspicious of the changes 
in the monitoring system at first, and that the role of 
the state was to ease anxieties and create an atmo- 
sphere more conducive to dialogue. For example, 
one participant commented: 

I think when we first started the districts were very 
defensive and I think that as our philosophy has 
changed that they realize that we’re all in this to- 
gether. We’ll come out to your district, do a free 
evaluation and if we can see some areas that you 
can’t see we’ll make some recommendations that 
will improve your program. 

Another participant from a different reiterated the 
above comments: 

Any time we institute something new there’s a lot 
of anxiety and a lot of questions regarding it and so 
we really try to put the districts at ease. Let them 
know that we’re not perfect, sometimes we make 
errors, we work on fixing them or revising them 
based on feedback. 

We asked participants to tell us what aspects 
of accountability reform in general education they 
thought were positive for students with disabilities. 
Most participants approved of the emphasis on out- 
comes for all students and the belief that inclusion 
in accountability systems was an incentive for im- 
provement. For example, this participant explained 
that including students with disabilities in account- 
ability reform not only focused them on results, but 
narrowed in on key results that enabled the state 
office of education to apply pressure consistently 
on school districts. Interestingly, this participant 
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points out that because the state makes the results 
on state assessments and other indicators publicly 
available, school districts cannot argue that improv- 
ing the performance of students with disabilities is 
an impossible task anymore: 

We are really focused on results, and on certain re- 
sults. And because we are coming at the districts 
from every angle we have, always on the same 
thing. That to me is what has really begun to make 
a difference. People know things they didn’t know 
before. And the commissioner [for education] be- 
fore he had really published this information in- 
cessantly, and at first it got everybody very angry, 
and then it kind of wore everybody down, and now 
what people are doing instead is responding. What 
we have done is try very hard to publicize what and 
who’s been effective. Because the more we do that, 
the less willing are people to stand up and say this 
isn’t fair and we can’t do this. 

Another participant from a different state shared 
the same view: “Until the last two years I don’t think 
that districts were really beginning to look at how 
students with disabilities are doing on state assess- 
ments and looking at data analysis. I think that has 
made a big difference. 

Several participants adopted a long-term view 
of the impact of accountability on students with dis- 
abilities. In their view the purpose of accountability 
reform was to improve the results and outcomes of 
education for all students, including those with dis- 
abilities. For example, one participant commented 
that the inclusion of students with disabilities in 
accountability reform had the potential to transform 
their whole educational experience by providing 
benchmarks as to the effectiveness of the system 
and defining the points where interventions would 
have the most impact: 

We want benchmarks of some kind to know that we 
are on the right track. But ultimately it is a matter 
of starting at the end and working backwards. What 



are the results that we want for kids; if it’s college, 
if it’s competitive employment with good pay and 
benefits and opportunities for advancement, re- 
ally meaningful work, whether its considered full 
participation, or partial participation the point is if 
that’s our goal we’ve got to work backwards and 
then everything that we do falls into line. I want to 
know how well our kids are doing in the recognized 
diploma, the recommended diploma, and the distin- 
guished diploma. One of the benchmarks to know 
if a kid’s on track for a recognized or distinguished 
diploma is whether or not they are taking algebra in 
8 th grade. The implications are what do you have to 
do before the kid gets to eighth grade to make sure 
that they will be in algebra. 



Likewise, another participant from a different 
state pointed out the necessity of using the informa- 
tion gained from accountability measures to achieve 
meaningful educational outcomes for students with 
disabilities. For this participant standards-based 
accountability reform meant increasing post school 
outcomes for students by improving graduation 
rates, rather than improving graduation rates as an 
end in itself: 



It has to be integrated in everything that we do, 
because whether we are talking about standards or 
raising expectations, in our minds what we are say- 
ing to school folks is yes diploma is important, yes 
learning is important, but what is the outcome you 
are looking for in the whole process. Yes, you think 
it ends in diploma, but you have to be thinking post 
school. That’s what we are pushing now, that stan- 
dards is a way to get the outcomes and we get better 
kids who are better prepared with an IEP diploma, 
better workers, better college kids with more op- 
portunities. 



One participant explained that she felt special 
education was now talking the same language as 
regular education and that this was very positive: 



I’m very optimistic about what’s going on. And the 
other thing that’s changed is that we got away from 
the procedural compliance to looking at improving 
outcomes. And so people were beginning to hear 
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that special education was talking the same lan- 
guage as the rest of them. Our new benchmarks and 
triggers have to do with performance, percent of 
kids who take the assessments, kids who graduate, 
drop-out rates, all that kind of thing. As a profes- 
sional in special education if you look at the results 
that we’ve had for our children as adults it’s abys- 
mal. So who cares if we did their IEP on time? 

We also asked state informants to discuss any 
concerns that they had regarding the inclusion of 
students with disabilities in accountability reform. 
Some participants expressed concern at the impact 
of state-level general education reforms that were 
targeted at the individual student level, specifically 
grade retention and high school graduation. They 
pointed out that in the past, students with disabili- 
ties may have been excused from the implications 
of such policies due to IEP team decisions, waiv- 
ers, differentiated or local diploma options, and 
IEP team exemption options. The potential loss of 
these protections when coupled with more rigorous 
exams caused concern for some participants. 

Only one of EPRRI’s participating states had 
adopted a policy of retention, which was being 
phased in over several years. Participants from this 
state expressed concern that the state’s promotion 
and retention policy could lead to an increase in the 
number of students referred to special education. 
One participant explained that although the state 
had implemented policies to support students at risk 
of failing and had established professional devel- 
opment initiatives in the lower grades, there was a 
risk that more students would be referred to special 
education. This participant perceived students from 
minority backgrounds to be at greater risk because 
they were viewed to be the ones who most likely 
would be failing due to a lack of instruction, rather 
than because they had a disability. 



The thing that got the most publicity is the social 
promotion piece: now if you don’t pass a test at 
a certain age you don’t get promoted to the next 
grade. There are a ton of safety nets built into it: A 
screening instrument, we are going to retool teach- 
ers, and provide accelerated reading instruction. I 
do worry. If you can visualize a snake and when 
snake eats a rat, and it travels down the body, right 
now the rat is somewhere between 4 th and 5 th grade 
as to when we get our referrals to special education. 
I don’t want that rate moving into second grade be- 
cause they’re afraid of the test and they can exempt 
them to the alternate and they’ll set a low perfor- 
mance standard, they’ll pass and then our numbers 
jump back up. I’m hoping we can prevent a run on 
special education due to the social promotion piece. 
Here’s the one group of kids I don’t want more of. 
I don’t want more kids that didn’t receive good in- 
struction and I don’t want more minorities simply 
because they didn’t receive that instruction. 



Participants in three out of four states voiced 
particular concern for those students in special edu- 
cation who fall into the “gray area.” In the words 
of one participant: “People talk about kids in the 
gray area, between the two tests.” These students 
were described as being too high functioning for the 
state’s alternate assessment, but not high function- 
ing enough for the state assessment. A participant 
from the same state explained : 



There’s these kids and you don’t know how many 
there are that people anticipate are going to be scor- 
ing very, very badly, but yet they shouldn’t take 
the alternate assessment, and the question will be, 
down the road, what do we want to do? I don’t 
see any of those kids moving into the alternate we 
have, the only ones we move in are the ones that 
should have been there in the first place. It’s not as 
a result of doing poorly on something else that you 
are going to slide into that. They’re very, very, very 
different assessment processes. 



A participant from another state expressed the 
firm belief that the number of “gray area” students 
would decrease as all students were provided with 
the opportunity to learn the curriculum. However, 
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as this participant pointed out there would still be 
a group of students with disabilities that would not 
do well: “Allowing children to have opportunities 
they haven’t had before, I think that will narrow 
the gray group down. I do believe that there will be 
kids who still will never make it through any level 
except maybe the lowest.” 

The issue was a particular concern at the high 
school exit level where performance on the state 
high school assessment was or would be a prerequi- 
site for a high school diploma. Policymakers were 
concerned that students with disabilities would not 
be able to graduate with regular diplomas. Three of 
EPRRI’s study states were considering some form 
of student level accountability at the high school 
exit level. The fourth already had a student level 
high stakes assessment at the exit level, but had 
temporarily made provision for students with dis- 
abilities. This “safety net” allowed students with 
disabilities to pass the high stakes examinations 
with a lower score than students without disabilities 
and graduate with a local diploma. A participant 
from this state commented: 

The question then is a policy question. Do we want 
every kid to have a way to get a local diploma, and 
therefore in effect to go back to layering if you will 
of expectations and state standards. That will be a 
very big issue. This begs the whole question about 
down the road, and this 1 think is still down the road 
question for us because we still have our safety net 
in place. 

In another state all students graduating in the 
2006 school year and thereafter would be required 
to take and pass the state high school exit examina- 
tion. One participant explained that students with 
disabilities who previously would have graduated 
with a high school diploma would not do so in the 
future: 



The biggest issue, in my opinion, is for that stu- 
dent that you’re talking because in past times that 
student would have normally matriculated through 
school, would have had a differential standard and 
would have received a high school diploma just 
like all the other students. Now we now have an- 
other group of students that potentially will not be 
eligible to receive a high school diploma that have 
received them in the past. 

Comments from another participant in the same 
state revealed concerns about how standards-based 
reform in his/her state would work in practice. This 
state informant posed the question of what would 
happen if large numbers of students fail to reach 
the required level of performance: 

We could discover that it’s very difficult to teach all 
children up to these standards and it would create 
an interesting situation. The intended consequence 
is that wonderful programs will be offered to all 
students and they’ll reach the bar. What if the won- 
derful program is offered to them and they don’t 
reach the bar? That’s an unintended consequence; 
there’s not supposed to a large number of students 
there. 

On the same issue a different participant from 
the same state pointed out that potentially high fail- 
ure rate at the exit level is not a special education 
issue: “If you have seen our statistics on the passing 
rate at the high school exit— we are facing a lot of 
pressure. I think it’s not going to be just special 
ed. that’s saying-gosh you are going to deny a lot 
of our kids a diploma. 

Meeting the Technical Challenges of Including 
Students with Disabilities in General 
Education Reform 

While states are testing more students with 
disabilities and offering a range of test accommo- 
dations and modifications, they face the issue of 
how to ensure that these assessments generate valid 
data. Issues of test validity and construct-relevance 
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underlie the decisions that states have made about 
who gets tested on what and how, whose test scores 
are reported and how, and whose scores are included 
in accountability measures. Participants identified 
several problems arising from attempts to include 
students with disabilities in general accountability 
systems. 

One participant described the contradictory 
nature of his state’s accountability system as a 
“one size fits all” that, once in place, had to be 
revised when it became apparent to a variety of 
stakeholders that this concept was incorrect. This 
participant explained that because children are 
complex individuals the only way to assess their 
progress meaningfully is to have a flexible system 
that makes provision for all learners. However, this 
flexibility becomes problematic in the sphere of 
accountability because the system needs uniform 
data and variables that measure the same construct 
over time: 

I think, there’s a goal implicit in all of the con- 
cerns - special education students, Title I students. 
There’s been an implicit there and there’s been an 
attempt to implement a system where one size fits 
all. And what the voices inside the department, and 
also the parents and people in the field who repre- 
sent Title I students or special education students 
or English Language Learners just to name three 
of them - have all pointed out the inflexibility of 
the initial notion, the naivete of the initial plan for 
a standards-based system, and therefore, the need 
for accommodations in the system to serve the 
complexity of what children are actually like. So 
flexibility is the issue, which - if this is going to im- 
prove outcomes effectively, that flexibility will be- 
come more and more built into it. For accountability 
purposes and your discussion here, the problem is 
that it’s much easier to use accountability numbers 
or data if one number always means the same thing. 
Flexibility works against accountability. You can 
put that in stone and the more ways in which your 
data is complex, the less you have control of the 
variables of education. So the accountability sys- 



tem really wants uniform data, but children are not 
uniform in their educational attainment. 

An area that was causing problems for sev- 
eral states was how to include the scores from the 
alternate assessment in the state’s accountability 
system, as required by NCLBA. To comply with 
IDEA 97 three of EPRRI’s four core study states 
had alternate assessments in place by the 2001- 
2002 school year and had begun to publicly report 
performance; while the fourth planned to have its 
alternate assessment in place by 2002-2003 school 
year. However, the technical challenge of whether 
to fold these data into the accountability systems 
and how to do this had yet to be solved. 

In one state, which we visited before NCLBA, 
several participants commented that the alternate 
assessment and the general assessment in their 
state were measuring very different skills and that 
more research was needed before any decision was 
made: 

The alternate assessment won’t show up in the state 
assessments, they will be a separate assessment re- 
port. As far as how they hold up in terms of the 
different accountability things that we have, I think 
that’s to be decided yet because we really want a 
couple of years to understand this thing. To be frank 
with you I think its probably true everywhere else, 
our level of understanding of this is not very high. 

I think what we are trying to do is to take a long- 
term view. We are trying to construct an intelligent 
response to it that will put us in a position to have 
good information before we make a final decision. 

I think that puts us in a good spot because a lot of 
people aren’t doing that. 

On the other hand, participants from states that 
had tied their alternate assessment to the overall 
state academic content standards commented that 
the alternate assessment would be merged into the 
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accountability system. However, they were less 
certain as to what this would look like in practice: 

The whole No Child Left Behind thing, all means 
all, and we are going to find a way to include spe- 
cial education children in the accountability sys- 
tem, even the severe and profound. There are the 
kids that just don’t take any test at all because at 
least in terms of the state assessment system, the 
alternative assessment is not ready yet. Eventually 
that will be folded in and reported. But we are just 
not there yet and I’m not sure what it is going to 
look like. 

Some state informants discussed the impact 
of assessment accommodations and the valid- 
ity of student scores. For example, at the time of 
our interviews one state used a norm referenced 
standardized assessment, but was in the process of 
phasing it out. One participant explained that the 
scores of students who took the assessment with a 
non standard accommodation were not included in 
the accountability index: 

Special education kids are expected to take it and 
most do. The question is whether you take it as 
a “standard administration,” that is within all the 
rules that are laid out by to define what a valid ad- 
ministration is. It is fairly restrictive; a reader, or 
calculator are non standard and if they are in a non- 
standard administration they are not part of the ac- 
countability system. 

Another participant explained further confirmed 
this: “If a student is tested with extra time or they 
have the test questions read to them or whatever, 
you really can’t compare their performance to that 
of the students in the norm group. So they’re not 
rolled into school summary data.” This informant 
then added: “I think they [the testing companies] 
just haven’t done the research, because you are 
talking about a very small numbers of kids and for 
them it just wasn’t worth doing.” 



The data also revealed concerns that the use 
of accommodations could conceal the true level 
of the students and could, for example, mask the 
fact that the student had not had the opportunity to 
learn basic skills. One participant explained that for 
assessments that did not have high stakes for stu- 
dents, it was important to look at where the student 
was compared to the state standards. According to 
this participant, who was discussing the English 
Language Arts accommodation policy, there were 
three complementary reasons for this policy: First, 
if districts know in advance that all scores will 
count in their rating, then they will intervene early 
in the child’s education. Second, the score gives 
more accurate information about the range of the 
students’ ability in reading and third it opens the 
door to academic intervention services: 



The 4 th and 8 th grade tests are not high stakes, they 
are for determining where kids in a given school 
are compared to standards. So if this kid can’t read 
certain sections that will show up as him not being 
able to read certain sections and that will be part of 
the score. And believe me, we had long discussions 
about this, but in the end, people felt, especially at 
4 th grade that if you allow districts to mask, if you 
will the kids actual functioning then two things can 
happen. One is that you’ll not know something be- 
cause you just won’t know what it would have been 
otherwise. Hopefully that takes away the incentive 
to get to that stage as you know those scores are 
going to count. Second that score opens up the gate 
to academic intervention services because it gives 
more accurate information about where the kid re- 
ally is to instructional staff and to the school. 



Participants from another state explained that 
they had students with disabilities who took an out 
of level assessment, but that the state did not know 
how to include out of level assessments in the ac- 
countability index. One participant described the 
then current discussions about out of level testing 
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and what the impact of these tests would have on 
the school’s accountability rating: 



piece that we’re kind of caught - How do we fix 
that? 



Now, our special education youngsters may be test- 
ed out of level, so that being the case, the student 
would get whatever performance standard his/her 
score fell into. You may have a fifth grader taking 
a second grade test, and their performance was ad- 
vanced. A piece that’s yet to be resolved in school 
accountability - how is that student’s score rolled 
in? It came up this morning and it has to do with 
special education students that are testing out of 
level and can we say that if you’re testing out of 
level, you automatically get a below basic or far 
below basic performance level? 



Other participants from the state pointed out, 
that districts faced a dilemma whatever policy was 
adopted. If schools knew that an out of level test 
automatically meant a below basic, they might 
instead administer a grade level test on the off 
chance that the student would score higher than a 
below basic. 



Well, if you think about it - if, in fact there was a 
rule that you take an out of level test you automati- 
cally receive a below basic score, then the district 
would say, “Why would I want to do that?” “I would 
rather have them take the in level because there 
may be a chance that they get a score that would 
enhance my opportunity to improve my API. 



Another participant suggested that if the policy 
adopted allowed out of level scores to count, then 
schools may be tempted to give out of level tests to 
students who did not need them as a way to improve 
their scores: 



I think the dilemma that we’re all in right now is the 
student who legitimately needs an out of level test 
to determine what he or she can do and what the 
next steps are. And then there are those students, 
they don’t need an out-of-level test to access the 
test. They’re capable of showing us what they know 
on a grade-level test and the perception in schools 
is - if I test them out of level, they’ll get a higher 
score and it’s going to help my school. That’s the 



Another participant from this state revealed that 
no final decision had been made and that the state 
was exploring alternative ways to make sure that 
schools were neither penalized nor tempted by out 
of level tests: 



I work with the Counsel for Chief State School Of- 
ficer Special Education Group, and one of the dis- 
cussions we’ve had there is that the student would 
get his or her full complement of scores so the par- 
ent would know where the child was performing at 
their instructional level; but that the school would 
receive perhaps a maximum amount - a maximum 
level they could receive for students who tested out 
of level. And, I don’t know. There are states that 
calibrate their alternates so that the kids get the 
same amount of credit as if they were taking a regu- 
lar test. There are a lot of questions. 
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Discussion 



These qualitative analyses provide insight into 
how over 35 state informants from 4 diverse states 
perceive the impact of accountability reforms on 
students with disabilities and the programs and 
systems that serve them. The results from inter- 
views and document analyses conducted in this 
study highlight the complexities involved in under- 
standing this issue. Examining the impact of these 
reforms requires not only exploring the multiple 
issues related to educational accountability and 
students with disabilities, but also requires an un- 
derstanding of the unique contexts internal to each 
state that influence individual perspectives. 

Although we conducted a small number of these 
interviews before the implementation of NCLBA, 
all participants were cognizant of its broad policy 
direction. Given the fact that NCLBA has radically 
altered the educational landscape, the following, 
discussion of the dominant themes emerging from 
the data is framed with reference to its major pro- 
visions and alignment with IDEA 97. Data from 
state informants indicates that they were balancing 
reform initiatives from three different directions: 1) 
Federal special education accountability reform; 
2) state general education accountability reform; 
and 3) federal general education accountability 
reform. 

Federal Special Education Reform 

Special education personnel were implementing 
the new outcomes based accountability require- 
ments of IDEA 97, which required them to collect 
and report data relating to performance and out- 
comes in addition to compliance data. Participants 
concerned with special education policy pointed out 
that IDEA 97’s new performance requirements and 
changes in federal monitoring of the implementa- 
tion of IDEA had fundamentally altered account- 



ability in special education. Participants agreed that 
the emphasis of accountability reforms on outcomes 
and data-driven accountability is a positive alterna- 
tive to the traditional process compliance system 
utilized in special education. Participants believed 
that the inclusion of students with disabilities in 
accountability reforms places the focus of special 
education effectiveness on educational outcomes 
and results and enables the state office of education 
to apply pressure consistently on school districts to 
ensure that students with disabilities are achieving 
state standards. 

State-level participants were very positive 
about the federal level shift away from process 
compliance monitoring to a new model that was 
results orientated and aligned with general educa- 
tion reform initiatives. However, the change from 
a process based to a results-based accountability 
system that encompasses participation and per- 
formance on state assessments has been a difficult 
process in some study states because it requires a 
paradigm shift. As a result, participants have found 
that some district and school level personnel have 
not included all students with disabilities in state 
assessment systems as required by IDEA 97. State- 
level participants listed 3 specific indicators related 
to IDEA participation requirements that they were 
monitoring closely - exemptions from state assess- 
ments, overrepresentation in alternate assessments, 
and rate of absences on testing days. 

Special education personnel were also getting 
used to the CIMP monitoring process from OSEP. 
The states that had already experienced the new 
approach were very positive about it. Those that 
had not were hopeful that the new approach would 
be an improvement on the old one. 
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State General Educational Accountability 
Reform 

The states included in this study already had a 
history of state mandated education reform in gen- 
eral education, but special education remained on 
the periphery of this. Special education personnel 
commented that in the past their department was not 
involved much in the planning and implementation 
of policy initiatives in general education. Many 
participants said that this situation had improved 
somewhat, but remained dependent on personal 
relationships within the state offices of education. 
Some state informants commented that special 
education personnel did not know how to analyze 
assessment data and make programmatic changes, 
while others were unsure of which students were 
included in the data. 

The debate on whether students with disabilities 
should be included in accountability systems in the 
same way as other student groups remains ongoing, 
but a key finding of this study was that most state 
informants wholeheartedly supported this policy 
direction. In one state data were already disaggre- 
gated by special education and regular education 
students and informants commented that this infor- 
mation provided evidence from state assessments 
that challenged the assumptions that students with 
disabilities would perform badly on state assess- 
ments and be unable to reach the standards required. 
When students with disabilities had the opportunity 
to participate participants reported that many educa- 
tors expressed surprise at the knowledge and skills 
demonstrated. 

However, in states with high stakes accountabil- 
ity at the individual student level, special education 
personnel and others were concerned that students 



with disabilities would suffer the consequences. 
Retention policies and graduation policies were 
specifically mentioned as causes for concern. 

Federal General Accountability Reform 

NCLBA has been called the most sweeping 
piece of federal education legislation in a generation 
and the results of this study support this opinion. 
In some ways, NCLBA provided the impetus to 
include students with disabilities that was lacking 
in state policy reforms, by requiring their participa- 
tion. The chief premise behind NCLBA is that every 
child can learn and that all children can achieve to 
high standards, regardless of race, socioeconomic 
status, or disability. By including students with 
disabilities as a target group, NCLBA challenges 
long held assumptions of the abilities of students 
with disabilities. Participants generally agreed that 
making students with disabilities count in account- 
ability systems was the only way to ensure that 
school systems pay attention to them and respond 
rapidly to their learning needs. However, all state 
informants pointed out that schools and school sys- 
tems could not continue business as usual in regard 
to the education of students with disabilities. 

A key finding across all states was the recogni- 
tion that special education and general education 
personnel at all levels of the system had increased 
their collaboration efforts and were working to- 
gether on an on-going basis to ensure that students 
with disabilities are integrated into all reform ef- 
forts. State-level personnel from all core study 
states stressed the importance of general education 
and special education working together and the 
necessity of involving special education person- 
nel in general education accountability reforms to 
integrate students with disabilities into the reform 
efforts. 
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Making sure that students with disabilities have 
the opportunity to learn the content standards from 
highly qualified teachers able to teach students with 
diverse needs was also identified as a key require- 
ment. Participants believed that the curriculum, 
state standards, and state assessments needed to 
be better aligned throughout the education system. 
Although the extent to which the state directed or 
approved the curriculum and instructional materials 
used by local school districts varied, participants 
voiced three common concerns regarding align- 
ment. 

First, some participants perceived that teachers 
remained unsure of how to use the state academic 
content standards to teach the required content in 
their classrooms and needed more guidance to rec- 
ognize student competencies associated with each 
level of achievement. Second, participants pointed 
out that it was essential to ensure that “backmap- 
ping” of academic content standards from grade 
to grade be carried out diligently because student 
knowledge was built up from year to year. Third, 
participants attributed concerns over the align- 
ment of instruction in the classrooms to the issue 
of educators and leadership capacity at the school 
and district level. 

Participants emphasized the importance of 
developing teacher and administrator capacity to 
effectively implement curricula at the school level, 
make use of the data collected, and realize the full 
potential of the accountability reforms. Participants 
were concerned that without appropriate profes- 
sional development for teachers and administrators 
at the school level, curriculum and instructional 
materials aligned across grades and designed to 
meet the unique needs of diverse learners in the 
classroom would not be effectively implemented. 



The role of the principal as instructional leader 
of the school emerged as a crucial element to the 
success of accountability reform. Participants firm- 
ly believed that good instructional leadership at the 
building level was essential to effectively deliver a 
standards-based curriculum as the principal had the 
responsibility to ensure that the state curriculum or 
state learning standards were taught effectively at 
each grade-level. Participants expressed concern 
about the shortage of school- and district-level ad- 
ministrative leaders. Informants commented that 
many principals did not have the time to talk to 
teachers about standards and curriculum changes 
and that this was detrimental to the success of the 
accountability reforms. 

The importance of improving teacher quality 
was perceived as essential to the success of ac- 
countability reform. Participants commented that 
all regular educators and special educators must 
have both content knowledge and the pedagogical 
skills to ensure that all students are able to access 
the curriculum and achieve the state standards. 
Because students with disabilities are included in 
the accountability reforms, there is an increasing 
need for all teachers to be able to work with diverse 
learners and expand their knowledge of instruc- 
tional strategies. 

In general, participants believed that both pre- 
service teacher training programs and professional 
development programs need to be redesigned to 
reflect the change in knowledge requirements 
resulting from accountability reform. Participants 
pointed out that beginning special educators now 
need specific content knowledge to support students 
with disabilities to ensure that they had the oppor- 
tunity to leam the general education curriculum 
regardless of placement or disability. In particular, 
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participants emphasized the importance of increas- 
ing beginning special educator knowledge and 
skills in mathematics and in reading. Participants 
also emphasized that beginning regular and special 
education teachers needed an array of pedagogical 
skills and instructional strategies for children with a 
range of abilities to ensure that students had access 
to the general curriculum. 

To address the need to increase teacher content 
and pedagogical knowledge, states included in 
this study have begun to redesign their pre-service 
teacher preparation programs and are implement- 
ing state-level initiatives to improve the quality 
of teachers by establishing more rigorous teacher 
standards for preparation and longer field experi- 
ences for pre-service programs. Under new federal 
higher education legislation, states are now able to 
influence the content and design of teacher prepara- 
tion programs and as a result, states are beginning to 
implement programs to increase the quality of the 
teacher education faculty and the programs offered 
to pre-service teachers. 

Additionally, as a means to increase teacher 
quality, many participants described initiatives de- 
signed to increase the state requirements for teacher 
licensure. State standards for teacher credentials 
for special and general education teachers are 
increasing and states are working to increase the 
rigor of teacher certification. However, there is a 
conflict between the need to increase teacher quality 
and the shortage of teachers. Thus, some states are 
working to implement innovative initiatives to try 
to attract new teachers and retain newly qualified 
teachers by recruiting at college and high school 
level. States are also attempting to recruit teachers 



from other states and working to developed partner- 
ships between the district level and universities to 
increase teacher capacity 

Participants pointed out that improving the 
knowledge and skills of practicing teachers was an 
equally serious issue. Some participants described 
how state-level professional development activities 
for in-service teachers were changing as a direct 
result of accountability reform. For example, one 
state now instructs districts on how to spend their 
state professional development funds and the num- 
ber of hours of professional development required 
for every number of years taught is often now state 
determined. State personnel had experienced some 
resistance from districts to providing the necessary 
release time for teachers to attend training courses 
at the educational service centers and participants 
felt there was a negative attitude of school boards 
of education toward professional development. 

A number of participants discussed state-de- 
veloped alternate certification programs that are 
shorter than traditional pre-service programs and 
provide an additional route for training of teachers. 
Participants who worked in states with alternate 
certification programs stressed the importance of 
involving the higher education institutions in these 
alternative programs so that stakeholders view the 
alternate program as comparable to a traditional 
program. Some of the participants in the study did 
express concerns regarding the development and 
implementation of alternate certification programs 
and cautioned that these programs should not be 
viewed as a panacea for the teacher shortage. 
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Participants in each of the states raised a num- 
ber of technical concerns relating to the develop- 
ment and administration of state assessments and 
the state’s capacity to track and report student 
enrollments and performance data. The effects of 
non-standard accommodations on the validity of 
the score for accountability purposes were prob- 
lematic for many participants. At the time of these 
interviews the scores of students who took the state 
assessment with a non standard accommodation 
could be excluded from the accountability system 
or counted as below basic. Several participants 
pointed out that the situation could be resolved in 
part through the creation of assessments that have 
been normed on students with disabilities and 
through careful development of state policies on the 
use of certain accommodations. Participant com- 
ments revealed that there were several arguments 
against the use of some accommodations on state 
assessments. They were concerned that the use 
of an accommodation during an assessment could 
conceal the “true” level of the students and mask the 
fact that the student performed badly not because 
they had a disability, but because he or she had not 
had the opportunity to learn basic skills. 

Participants also expressed concern about the 
validity, reliability, and construct-relevance of the 
alternate assessment compared to the general state 
assessment. Several participants did express con- 
cern about the tension relating to including students 
with disabilities in the assessments at the school 
level and the controversy regarding the validity of 
the score of an alternate assessment and how the 
score will be included in the accountability system. 



Three out of four of the EPRRI core study states 
had alternate assessments in place by 2001-2002. 
A variety of views were shared by participants with 
some arguing that the alternate and standard test 
are measuring very different skills and that more 
research is needed before a decision about includ- 
ing the scores could be made. Other participants 
commented that their state’s alternate assessments 
were tied to the overall state academic content 
standards and therefore, should be merged into the 
accountability system. 
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Conclusions 



These analyses contribute to the current knowl- 
edge base on the impact of concurrent accountabil- 
ity reforms that emanate from both the state and 
federal level. Clearly, however, NCLBA is now 
the dominant force in educational policy for all 
students. While participants believed in the over- 
all purpose of including students with disabilities 
in the accountability reforms, numerous questions 
arose throughout the interviews regarding how the 
educational systems in individual states will be 
able to overlay NCLBA accountability policy and 
fulfill the new requirements of this reform within 
an already established state system. Issues ranged 
from educator and administrator capacity to testing 
development and accommodations. Solutions for 
many of these issues are currently being explored 
and state participants shared many new alternate 
program ideas created to address the unique issues 
relating to accountability reforms. Tension clearly 
exists between NCLBA requirements, state require- 
ments, and current IDEA 97 requirements. Issues 
such as the extent to which the IEP team is in con- 
trol of the educational decision-making process of 
students with disabilities, the requirement for highly 
qualified teachers and how it relates to special edu- 
cators, and the impact of accommodations on the 
validity of scores on state assessments remain to be 
solved in both policy and practice. 
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